Jais-3-3.7B
Jais-3-3.7B is the small dense member of the Jais-3 family: a 3.7B-class decoder-only model with a 512K context window.
[!TIP] Other formats are available in the IFM/Jais-3 collection:
- Quantization: IFM/Jais-3-3.7B-FP8
- GGUF for llama.cpp and Ollama: IFM/Jais-3-3.7B-GGUF
- MLX for Apple silicon:
Benchmark Results
| Reference models | |||||
|---|---|---|---|---|---|
| Jais-3-3.7B | Qwen3.5-4B | Gemma 4 E2B | granite-4.0-h-3b-ar | Falcon-H1-3B-Instruct | |
| # Params | 3.7B | 4B | 5B | 3B | 3B |
| # Activated params | 3.7B | 4B | 2.3B | 3B | 3B |
| Architecture | Dense | Dense | Dense | Dense | Dense |
| Open-ended Generation | |||||
| AraGen | |||||
| AraGen v3 (3C3H)09-2025 | 49.6 | 45.8 | 39.6 | 27.1 | 13.0 |
| Correctness | 57.9 | 57.2 | 46.5 | 30.8 | 16.8 |
| Completeness | 56.9 | 56.3 | 46.4 | 30.4 | 15.3 |
| Conciseness | 18.2 | 10.3 | 11.5 | 14.4 | 3.5 |
| Helpfulness | 51.9 | 45.5 | 41.5 | 27.6 | 13.4 |
| Honesty | 55.6 | 49.0 | 45.2 | 28.9 | 13.8 |
| Harmlessness | 56.9 | 56.3 | 46.5 | 30.6 | 15.4 |
| Knowledge & Reasoning | |||||
| General Knowledge & Comprehension | |||||
| Arabic MMLU*Accuracy | 83.3 | 75.2 | 69.4 | 59.8 | 51.0 |
| BelebeleAccuracy | 91.9 | 84.0 | 79.6 | 61.9 | 57.9 |
| STEM & Math | |||||
| AraSTEM*Accuracy | 91.7 | 91.4 | 85.1 | 66.5 | 64.9 |
| Arabic GSM8K*Accuracy | 91.9 | 88.7 | 81.7 | 67.2 | 72.2 |
| GATmath*Accuracy | 96.2 | 95.9 | 90.5 | 45.3 | 41.3 |
| Professional Domains | |||||
| MedAraBench*Accuracy | 74.1 | 74.1 | 57.2 | 38.1 | 34.5 |
| Arabic Legal (HELM)*Accuracy | 66.7 | 63.3 | 57.1 | 56.1 | 53.0 |
| Arabic Language & Culture | |||||
| Islamic Knowledge & Arab Culture | |||||
| IslamicQAAccuracy (mean of the 4 below) | 81.7 | 61.6 | 63.0 | 41.2 | 48.2 |
| PaLM-X 2025-2*Accuracy | 88.9 | 79.6 | 79.2 | 57.5 | 66.6 |
| QIAS-1*Accuracy | 83.3 | 50.7 | 51.6 | 18.2 | 27.5 |
| QIAS-2*Accuracy | 80.0 | 69.9 | 65.3 | 66.5 | 54.3 |
| IslamicEval2025-1B*Accuracy | 74.5 | 46.3 | 55.8 | 22.5 | 44.6 |
| Arab CultureAccuracy (mean of the 3 below) | 81.2 | 68.8 | 66.8 | 60.2 | 54.8 |
| ArabCulture*Accuracy | 84.3 | 76.2 | 74.7 | 68.4 | 64.2 |
| AraDiCE-Culture*Accuracy | 76.5 | 61.5 | 56.4 | 53.6 | 45.8 |
| Arabic Cuisine^Accuracy | 82.6 | 68.9 | 69.2 | 58.6 | 54.4 |
| Dialects | |||||
| Dialectal Arabic MMLU*Accuracy | 81.6 | 76.5 | 65.1 | 50.7 | 43.8 |
| Dialect Identification (QADI)Accuracy | 68.8 | 37.4 | 44.4 | 42.2 | 28.5 |
| Alyah*Accuracy | 79.7 | 63.3 | 69.7 | 66.0 | 58.9 |
| Linguistics & Literature | |||||
| Arabic MMLU with diacriticsAccuracy | 80.7 | 70.8 | 64.7 | 53.3 | 34.6 |
| FannOrFlop-MCQAccuracy | 66.2 | 50.9 | 60.0 | 48.8 | 41.1 |
| Arabic Linguistics^Accuracy (mean of the 2 below) | 92.1 | 72.7 | 77.9 | 49.3 | 42.1 |
| DiacritizationAccuracy | 86.9 | 79.2 | 74.2 | 64.1 | 51.2 |
| ParsingAccuracy | 97.3 | 66.3 | 81.6 | 34.6 | 32.9 |
| Assistant Capabilities | |||||
| Instruction Following | |||||
| Arabic IFBench^Accuracy (prompt-level strict) | 92.6 | 53.7 | 37.6 | 14.7 | 18.4 |
| Arabic IFEvalAccuracy | 79.0 | 73.0 | 69.6 | 43.1 | 55.7 |
| Factuality & Hallucination | |||||
| HalluScoreAccuracy | 48.3 | 37.5 | 33.3 | 16.4 | 13.2 |
| SimpleQA Verified (Ar)Aggregate (2/3 F-score + 1/3 non-hallucination rate) | 29.1 | 7.0 | 29.8 | 3.5 | 2.5 |
| F-score | 1.5 | 7.4 | 2.2 | 4.5 | 2.3 |
| Non-hallucination rate | 84.2 | 6.3 | 84.8 | 1.4 | 3.0 |
| Agentic | |||||
| Terminal Bench 2.1Accuracy (Avg@3) | 41.2 | 25.8 | 0.4 | 0.0 | 0.0 |
| GDPValAccuracy (@ 75% threshold) | 54.5 | 15.0 | 3.2 | 0.0 | 0.0 |
| MCPMarkAccuracy (Pass@1) | 30.7 | 25.2 | 1.6 | 0.0 | 0.0 |
| WildClawBenchMean Score@3 | 40.9 | 33.3 | 18.2 | 8.2 | 0.1 |
| Apex-AgentsAccuracy (Pass@1) | 16.7 | 6.4 | 0.0 | 0.5 | 0.2 |
| GAIA v2 lit ARAccuracy (Pass@1) | 45.5 | 27.9 | 18.2 | 0.0 | 0.0 |
| BrowseComp-PlusAccuracy | 64.8 | 50.6 | 16.4 | 0.6 | 0.0 |
[!NOTE] Starred (*) benchmarks have been audited for sample quality. Items with ambiguous questions, invalid answer options, or indefensible gold labels were excluded, since they can penalize a correct answer or reward a lucky guess. Scores on these benchmarks are reported on the cleaned sets. The audit methodology, the excluded samples and the verified benchmarks are available in the IFM/Arabic-Benchmark-Auditing collection.
[!NOTE] Benchmarks marked with ^ will be released by IFM soon.
Quickstart
Serving
vLLM (vllm>=0.30.0) · vLLM recipe:
vllm serve inception42/Jais-3-3.7B \
--trust-remote-code \
--dtype bfloat16 \
--tensor-parallel-size 1 \
--reasoning-parser k2_horizon \
--enable-auto-tool-choice \
--tool-call-parser k2_horizon
Use an exact branch name from the inventory with vLLM's --revision option.
SGLang (sglang>=0.5.20) · SGLang recipe:
sglang serve \
--model-path inception42/Jais-3-3.7B \
--tp 1 \
--dtype bfloat16 \
--attention-backend fa3 \
--reasoning-parser k2_horizon \
--host 0.0.0.0 \
--port 30000 \
--tool-call-parser k2_horizon
API Usage
[!TIP] Recommended settings:
reasoning_effort="high", the sampling hyperparameters in the table below, and at least 32,768 output tokens. Reasoning depth is selected per request throughchat_template_kwargs. Thinking is returned inreasoning_contentand the answer incontent.
| Parameter | Default | Thinking mode, English | Thinking mode, Arabic |
|---|---|---|---|
temperature |
0.2 | 1.0 | 0.7 |
top_p |
0.95 | 0.95 | 0.95 |
top_k |
20 | 20 | 20 |
from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="inception42/Jais-3-3.7B",
messages=[{"role": "user", "content": "لدينا عددان مجموعهما 50 وحاصل ضربهما 600. ما هما العددان؟ اشرح خطوات الحل."}],
temperature=0.7,
top_p=0.95,
max_tokens=32768,
extra_body={"top_k": 20, "chat_template_kwargs": {"reasoning_effort": "high"}},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)
import json
from datetime import datetime
from zoneinfo import ZoneInfo
from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
TIMEZONES = {"أبوظبي": "Asia/Dubai", "الرياض": "Asia/Riyadh", "القاهرة": "Africa/Cairo", "لندن": "Europe/London"}
def get_current_time(city: str) -> dict:
if city not in TIMEZONES:
return {"error": f"مدينة غير معروفة: {city}"}
return {"city": city, "time": datetime.now(ZoneInfo(TIMEZONES[city])).strftime("%H:%M")}
tools = [{
"type": "function",
"function": {
"name": "get_current_time",
"description": "يعيد الوقت الحالي في مدينة معينة",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string", "description": "اسم المدينة بالعربية"}},
"required": ["city"],
},
},
}]
available_tools = {"get_current_time": get_current_time}
messages = [{"role": "user", "content": "كم الساعة الآن في القاهرة؟"}]
for _ in range(5): # call the model, run any requested tools, repeat until it answers
message = client.chat.completions.create(
model="inception42/Jais-3-3.7B",
messages=messages,
tools=tools,
temperature=0.7,
top_p=0.95,
max_tokens=32768,
extra_body={
"top_k": 20,
"chat_template_kwargs": {"reasoning_effort": "high", "tool_call_format": "xml"},
},
).choices[0].message
messages.append(message)
if not message.tool_calls:
break
for call in message.tool_calls:
tool = available_tools.get(call.function.name)
arguments = json.loads(call.function.arguments)
result = tool(**arguments)
messages.append({"role": "tool", "tool_call_id": call.id,
"content": json.dumps(result, ensure_ascii=False)})
print(message.content)
Our model supports multiple tool-call formats, which can be changed with chat_template_kwargs. The supported values are json, xml, and xml_typed. The default is xml. Keep --tool-call-parser k2_horizon enabled to parse the selected format.
Transformers
Validated with Transformers 5.15.0, PyTorch 2.13.0, Safetensors 0.8.0.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "inception42/Jais-3-3.7B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
)
inputs = tokenizer("لدينا عددان مجموعهما 50 وحاصل ضربهما 600. ما هما العددان؟ اشرح خطوات الحل.", return_tensors="pt").to(model.device)
inputs.pop("token_type_ids", None)
outputs = model.generate(**inputs, max_new_tokens=32768, temperature=0.7, top_p=0.95, top_k=20, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Release Artifacts
The table below lists the release artifacts for Jais-3-3.7B.
Artifact Index
| Artifact | Link |
|---|---|
| Model card | Hugging Face |
| Training logs | W&B |
| Blog post | Jais 3: Introducing Arabic-centric Agentic AI |
| Checkpoints | Checkpoint inventory |
| Technical report | Jais 3 Technical Report |
| Code repository | Coming soon |
Checkpoint Inventory
Model repository: inception42/Jais-3-3.7B
Branch names below refer to this repository.
| Checkpoint | Branch / repository |
|---|---|
| CPT Checkpoint | CPT |
| Cooldown Checkpoint | Cooldown |
| FTPO Checkpoint | FTPO |
| SFT Checkpoint | SFT |
Best Practices
- Reasoning effort: always
high. All reported results use high reasoning effort. Pass{"chat_template_kwargs": {"reasoning_effort": "high"}}on every request. No reasoning ({"reasoning_effort": "low"}) is not recommended except for research on reasoning efforts. - Sampling parameters. Use the values in the table above:
temperature=0.2,top_p=0.95,top_k=20by default. In thinking mode, settemperature=1.0for English andtemperature=0.7for Arabic. Keepmin_p=0.0,presence_penalty=0.0andrepetition_penalty=1.0. - Serving. Use the validated SGLang recipe above: BF16, TP=1 on a single H100 node, FlashAttention-3.
- Parsers. Enable the
k2_horizonreasoning parser for chat, and add thek2_horizontool-call parser for agent use. Leave both off for plain completion-style generation.
Intended Use and Limitations
Intended use: The model is designed for Arabic and English assistant and agentic applications, including reasoning and tool-use tasks, as well as for research and experimentation.
Out of scope: The model should not be used in ways that violate applicable laws or regulations. The following uses are also considered out of scope:
- Harmful or malicious use: generating hateful, extremist, discriminatory, or deliberately misleading content, or using the model to support illegal or harmful activities.
- Sensitive information: using the model to generate, expose, or infer personal, confidential, or otherwise sensitive information about individuals or organizations.
- Unsupported languages: applications that depend on reliable performance in languages other than Arabic or English.
- High-stakes decisions: relying on the model to make medical, legal, financial, or safety-critical decisions without appropriate human oversight.
Limitations: The model can produce inaccurate or made-up content and may reflect biases present in its training data. Evaluate it for your use case and add safeguards before deploying it.
Citation
@misc{jais3_2026,
title = {Jais 3: Introducing Arabic-centric Agentic AI},
author = {{IFM Team, Inception42 Team, and Cerebras Team}},
year = {2026},
url = {https://ifm.ai/jais/tech-report},
}