GritLM/GritLM-7B (Quantized)

Description

This model is a quantized version of the original model GritLM/GritLM-7B.

It's quantized using the BitsAndBytes library to 4-bit using the bnb-my-repo space.

Quantization Details

• Quantization Type: int4
• bnb_4bit_quant_type: nf4
• bnb_4bit_use_double_quant: True
• bnb_4bit_compute_dtype: bfloat16
• bnb_4bit_quant_storage: uint8

📄 Original Model Information

Model Summary

GritLM is a generative representational instruction tuned language model. It unifies text representation (embedding) and text generation into a single model achieving state-of-the-art performance on both types of tasks.

• Repository: ContextualAI/gritlm
• Paper: https://arxiv.org/abs/2402.09906
• Logs: https://wandb.ai/muennighoff/gritlm/runs/0uui712t/overview
• Script: https://github.com/ContextualAI/gritlm/blob/main/scripts/training/train_gritlm_7b.sh

Use

The model usage is documented here.

Citation

@misc{muennighoff2024generative,
      title={Generative Representational Instruction Tuning}, 
      author={Niklas Muennighoff and Hongjin Su and Liang Wang and Nan Yang and Furu Wei and Tao Yu and Amanpreet Singh and Douwe Kiela},
      year={2024},
      eprint={2402.09906},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}