GGUF Weights for Jais-2-70B-Chat

This repository contains GGUF quantized versions of Jais-2-70B-Chat for use with llama.cpp and compatible inference engines.

Available Formats

Usage with llama.cpp

# Basic inference
cd ~/Jais-2-70B-Chat-GGUF/
llama-cli -m Q4_K_M.gguf -p "ما هي عاصمة الإمارات؟"

Hardware Requirements

Notes

• Recommended format: Q4_K_M for best quality/size tradeoff
• K-quant variants (Q3_K_M, Q4_K_M, Q5_K_M): Require -b 8 flag when using GPU offloading

---

Original Model Card

Jais-2: The Next Generation of Arabic Frontier LLMs

Model Overview

Jais-2-70B-Chat is a high-capacity bilingual Arabic–English language model developed by MBZUAI, Inception, and Cerebras. Jais-2-70B-Chat Model is trained from scratch on Arabic and English data and is powered by a custom Arabic-centric vocabulary, it efficiently captures Modern Standard Arabic, regional dialects, and mixed Arabic–English code-switching. The model is openly available under a Apache 2.0 license and also deployed as a fast, production-ready chat experience running on Cerebras hardware. Visit the Jais-2 Web App.

Key Technical Specifications

• Model Developers: MBZUAI, Inception, Cerebras.
• Languages: Arabic (MSA & dialects) and English
• Architecture: Transformer-based, Decoder-only architecture with multi-head self-attention.
• Parameters: 70 Billion
• Context Length: 8,192
• Vocabulary Size: 150,272
• Training Infrastructure: Optimized for Cerebras CS-2 and Condor Galaxy clusters
• Key Design Choices: Rotary Position Embeddings (RoPE), Squared-ReLU activation, custom μP parameterization, and 8:1 filter-to-hidden size ratio.

---

How to Use the Model

Using Transformers

1. Clone the Jais 2–compatible Transformers fork

# Pull the latest version and ensures you have the most up-to-date features/models and bug fixes.
# Note: could be not as stable as an official PyPI release.
uv pip install git+https://github.com/huggingface/transformers.git

2. Load and Inference on the Model

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the model and tokenizer
model_name = "inceptionai/Jais-2-70B-Chat"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

# Example Arabic prompt
system_prompt = "أجب باللغة العربية بطريقة رسمية وواضحة."
user_input = "ما هي عاصمة الإمارات؟"

# Apply chat template (always)
chat_text = tokenizer.apply_chat_template(
    [
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_input}
    ],
    tokenize=False,
    add_generation_prompt=True
)

# Tokenize and generate
inputs = tokenizer(chat_text, return_tensors="pt").to(model.device)
inputs.pop("token_type_ids", None)
outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)

# Decode and print
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
#عاصمة الإمارات العربية المتحدة هي أبوظبي.

Using vLLM

1. Clone the Jais 2–compatible vLLM fork

# Pull the latest version and ensures you have the most up-to-date features/models and bug fixes.
# Note: could be not as stable as an official PyPI release.
uv pip install git+https://github.com/vllm-project/vllm.git

2. Load and Inference on the Model

from vllm import LLM, SamplingParams

# Load model and tokenizer
model_name = "inceptionai/Jais-2-70B-Chat"
llm = LLM(model=model_name, tensor_parallel_size=1)
tokenizer = llm.get_tokenizer()

# Example Arabic prompt
system_prompt = "أجب باللغة العربية بطريقة رسمية وواضحة."
user_input = "ما هي عاصمة الإمارات؟"

# Apply chat template (always)
chat_text = tokenizer.apply_chat_template(
    [
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_input}
    ],
    tokenize=False,
    add_generation_prompt=True
)

# Run generation
sampling_params = SamplingParams(max_tokens=8192, temperature=0)
outputs = llm.generate([chat_text], sampling_params)

#Print output
print(outputs[0].outputs[0].text)
#عاصمة الإمارات العربية المتحدة هي أبوظبي.

Or serve through command line (CLI)

vllm serve inceptionai/Jais-2-70B-Chat \
    --served-model-name inceptionai/Jais-2-70B-Chat-Local --dtype bfloat16 \
    --tensor-parallel-size 2 --max-model-len 8192 --max-num-seqs 256 \
    --host 0.0.0.0 --port 8042 --api-key "Optional"

---

Evaluation

Performance Overview

We evaluate Jais-2-70B across two key benchmarks that capture both instruction following and generative Arabic ability: IFEval (English and Arabic) and AraGen-12-24 (3C3H).

IFEval Results (Strict 0-shot)

---

AraGen-12-24 (3C3H) Results

Overall, our results show that:

• Jais-2-70B delivers competitive Arabic and English instruction-following performance across IFEval metrics.
• Jais-2-70B achieves the highest scores across nearly all AraGen metrics, outperforming Qwen2.5-72B and Llama-3.3-70B on Arabic generative tasks.

---

Intended Use

Target Audiences

• Academics: Researchers focusing on Arabic NLP, multilingual modeling, or cultural alignment
• Businesses: Companies targeting Arabic-speaking markets
• Developers and ML Engineers: Integrating Arabic language capabilities into applications and workflows

Appropriate Use Cases

• Research:

  • Natural language understanding and generation tasks
  • Conducting interpretability or cross-lingual alignment analyses
  • Investigating Arabic linguistic or cultural patterns

• Commercial Use:

  • Building chat assistants for Arabic-speaking audiences
  • Performing sentiment and market analysis in regional contexts
  • Summarizing or processing bilingual Arabic–English documents
  • Creating culturally resonant Arabic marketing and entertainment content for regional audiences

Inappropriate Use Cases

• Harmful or Malicious Use:

  • Producing hate speech, extremist content, or discriminatory language
  • Creating or spreading misinformation or deceptive content
  • Engaging in or promoting illegal activities

• Sensitive Information:

  • Handling or generating personal, confidential, or sensitive information
  • Attempting to infer, reconstruct, or guess sensitive information about individuals or organizations

• Language Limitations:

  • Applications requiring strong performance outside Arabic or English languages

• High-Stakes Decisions:

  • Making medical, legal, financial, or safety-critical decisions without human oversight

Citation

If you find our work helpful, please give us a cite.

@misc{anwar2026jais2familyarabiccentric,
      title={Jais 2: A Family of Arabic-Centric Open Large Language Models}, 
      author={Mohamed Anwar and Abed Alhakim Freihat and George Ibrahim and Mostafa Awad and Abdelrahman Sadallah and Gurpreet Gosal and Gokulakrishnan Ramakrishnan and Sarath Chandran and Biswajit Mishra and Rituraj Joshi and Ahmed Frikha and Etienne Goffinet and Abhishek Maiti and Ali El Filali and Sarah AlBarri and Samujjwal Ghosh and Rahul Pal and Parvez Mullah and Awantika Shukla and Sajid siddiki and Samta Kamboj and Onkar Pandit and Sunil Kumar Sahu and AbdelRahman Elbadawy and Amr Mohamed and Ahmad Chamma and Evan Dufraisse and Abdelaziz Bounhar and Dani Bouch and Hadi Abdine and Guokan Shang and Fajri Koto and Yuxia Wang and Zhuohan Xie and Ali Mekky and Rania Elbadry and Sarfraz Ahmad and Momina Ahsan and Omar El Herraoui and Daniil Orel and Hasan Iqbal and Kareem Elzeky and Mervat Abassy and Kareem Elozeiri and Saadeldine Eletter and Farah Atif and Nurdaulet Mukhituly and Haonan Li and Xudong Han and Aaryamonvikram Singh and Zainul Abedien Ahmed Quraishi and Neha Sengupta and Larry Murray and Avraham Sheinin and Joel Hestness and Natalia Vassilieva and Hector Xuguang Ren and Zhengzhong Liu and Michalis Vazirgiannis and Preslav Nakov},
      year={2026},
      eprint={2608.13580},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2608.13580}, 
}