Sarvam-105B-1711 uses the same base model as Sarvam-105B, a 105B-parameter mixture-of-experts model with 10.3B active parameters, and is further post-trained to improve:
This repo has the NVFP4 build; the BF16 weights are at sarvamai/sarvam-105b-1711-bf16.
| Benchmark | Sarvam-105B-1711 (BF16) |
Sarvam-105B-1711 (NVFP4) |
Gemma-4-31B | Nemotron-Super-120B | Mistral-Small-4 | GPT-5.6 Luna | Claude Haiku 4.5 | Gemini 3.5 Flash-Lite |
|---|---|---|---|---|---|---|---|---|
| Instruction following | ||||||||
| IFEval | 96.5 | 96.4 | 94.7 | 90.8 | 82.3 | 93.1 | 86.1 | 95.6 |
| IFBench | 82.5 | 81.8 | 77.8 | 70.9 | 41.0 | 64.1 | 45.5 | 73.5 |
| MultiChallenge | 62.2 | 62.5 | 63.6 | 50.8 | 30.6 | 61.6 | 49.8 | 64.8 |
| Chat | ||||||||
| Arena-Hard v2 | 76.6 | 75.3 | 75.0 | 73.5 | 54.6 | 79.8 | 60.9 | 77.2 |
| WritingBench | 86.5 | 86.6 | 79.2 | 87.3 | 85.5 | 84.4 | 83.2 | 80.7 |
| Tool calling | ||||||||
| τ²-Bench | 85.5 | 84.1 | 81.1 | 77.5 | 55.3 | 79.5 | 62.1 | 83.1 |
| BFCL v4 | 62.4 | 62.7 | 68.8 | 55.4 | 55.0 | 67.7 | 68.6 | 67.8 |
| Agentic | ||||||||
| GDPval-S | 66.9 | 65.2 | 61.1 | 56.8 | 49.2 | – | 66.2 | 73.0 |
| Indic | ||||||||
| Indic Suite | 79.1 | 78.8 | 78.6 | 39.5 | 44.3 | 82.4 | 61.3 | 79.7 |
| Knowledge | ||||||||
| MMLU-Pro | 85.2 | 85.0 | 88.1 | 85.1 | 80.6 | 88.1 | 83.0 | 88.1 |
| GPQA Diamond | 80.4 | 78.2 | 84.3 | 78.9 | 71.0 | 88.7 | 67.9 | 85.2 |
| Humanity's Last Exam | 12.3 | 11.4 | 22.1 | 17.3 | 9.6 | 35.8 | 10.1 | 17.9 |
| Search | ||||||||
| BrowseComp | 41.4 | 35.0 | 33.2 | 27.9 | 14.5 | 76.3 | 18.4 | 42.9 |
| Math | ||||||||
| AIME 2025 | 87.9 | 86.0 | 85.2 | 85.5 | 81.5 | 99.2 | 83.9 | 73.5 |
| AIME 2026 | 89.7 | 88.0 | 89.0 | 87.6 | 87.1 | 97.2 | 86.3 | 83.2 |
| HMMT Feb 2025 | 78.0 | 77.0 | 88.8 | 85.7 | 73.8 | 99.5 | 74.4 | 73.0 |
| HMMT Nov 2025 | 81.5 | 79.7 | 88.0 | 84.1 | 81.4 | 99.1 | 82.6 | 78.4 |
| HMMT Feb 2026 | 71.8 | 70.1 | 77.3 | 76.4 | 74.4 | 93.0 | 67.5 | 64.8 |
| Coding | ||||||||
| LiveCodeBench v6 | 75.3 | 74.7 | 81.9 | 75.5 | 68.4 | 85.0 | 57.0 | 73.9 |
All scores are from our own runs; see Benchmark details.
DSpark with a fixed 5 draft tokens and probabilistic drafting, temperature 1.0, thinking on, FP8 KV cache, prefix cache cleared before each concurrency level, on a diverse set of reasoning prompts.
One B200 GPU, 128k context.
| Concurrent users | Total throughput (tok/s) | Speed per user (tok/s) | Time to first token, p50 (ms) | Time to first token, p90 (ms) |
|---|---|---|---|---|
| 1 | 532 | 576 | 63 | 68 |
| 8 | 1,978 | 293 | 82 | 199 |
| 16 | 2,500 | 229 | 86 | 266 |
| 32 | 4,311 | 174 | 131 | 471 |
| 64 | 5,354 | 122 | 156 | 841 |
| 128 | 7,490 | 80 | 211 | 1,436 |
One RTX PRO 6000 Blackwell GPU (96 GB), 50k context.
| Concurrent users | Total throughput (tok/s) | Speed per user (tok/s) | Time to first token, p50 (s) | Time to first token, p90 (s) |
|---|---|---|---|---|
| 1 | 30.9 | 61.8 | 8.1 | 8.1 |
| 4 | 43.9 | 14.4 | 8.5 | 8.9 |
| 8 | 54.9 | 12.2 | 19.5 | 40.5 |
Tested with vLLM 0.33.0.
export VLLM_USE_V2_MODEL_RUNNER=1
export VLLM_USE_BREAKABLE_CUDAGRAPH=1
export VLLM_ENFORCE_STRICT_TOOL_CALLING=0
export FLASH_ATTENTION_CUTE_DSL_CACHE_ENABLED=1
CAPTURE_SIZES='[1,2,4,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120,128,136,144,152,160,168,176,184,192,200,208,216,224,232,240,248,256,272,288,304,320,336,352,368,384,400,416,432,448,464,480,496,512,544,576,608,640,672,704,736,768,800,832,864,896,928,960,992,1024,1088,1152,1216,1280,1344,1408,1472,1536,1600,1664,1728,1792,1856,1920,1984,2048,2176,2304,2432,2560,2688,2816,2944,3072,3200,3328,3456,3584,3712,3840,3968,4096,4352,4608,4864,5120,5376,5632,5888,6144,6400,6656,6912,7168,7424,7680,7936,8192]'
MODEL_DIR="$PWD/models/sarvam-105b-1711"
hf download sarvamai/sarvam-105b-1711 --local-dir "$MODEL_DIR"
vllm serve "$MODEL_DIR" \
--trust-remote-code \
--served-model-name sarvam-105b \
--tensor-parallel-size 1 \
--max-model-len 131072 \
--max-num-seqs 256 \
--max-num-batched-tokens 8192 \
--gpu-memory-utilization 0.92 \
--kv-cache-dtype fp8_e4m3 \
--enable-auto-tool-choice \
--tool-call-parser glm45 \
--reasoning-parser glm45 \
--enable-prompt-tokens-details \
--speculative-config '{"method":"dspark","model":"'"$MODEL_DIR"'/dspark","num_speculative_tokens":5,"draft_sample_method":"probabilistic","rejection_sample_method":"block"}' \
--compilation-config '{"use_inductor_graph_partition":false,"pass_config":{"fuse_rope_kvcache_cat_mla":true},"cudagraph_capture_sizes":'"$CAPTURE_SIZES"',"max_cudagraph_capture_size":8192}'
vllm serve sarvamai/sarvam-105b-1711 \
--trust-remote-code \
--served-model-name sarvam-105b \
--tensor-parallel-size 1 \
--max-model-len 131072 \
--max-num-seqs 256 \
--max-num-batched-tokens 8192 \
--gpu-memory-utilization 0.92 \
--kv-cache-dtype fp8_e4m3 \
--enable-auto-tool-choice \
--tool-call-parser glm45 \
--reasoning-parser glm45 \
--enable-prompt-tokens-details \
--compilation-config '{"use_inductor_graph_partition":false,"pass_config":{"fuse_rope_kvcache_cat_mla":true},"cudagraph_capture_sizes":'"$CAPTURE_SIZES"',"max_cudagraph_capture_size":8192}'
Recommended temperature: 1.0 with thinking on, 0.6 with thinking off.
Thinking is on by default. Turn it off per request with chat_template_kwargs:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
messages = [{"role": "user", "content": "What is the capital of India?"}]
# Thinking on (default)
resp = client.chat.completions.create(
model="sarvam-105b",
messages=messages,
temperature=1.0,
)
print(resp.choices[0].message.reasoning) # thinking
print(resp.choices[0].message.content) # answer
# Thinking off
resp = client.chat.completions.create(
model="sarvam-105b",
messages=messages,
temperature=0.6,
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(resp.choices[0].message.content)
max for GPT-5.6 Luna and Claude Haiku 4.5 and high for Gemini 3.5 Flash-Lite. All scores are in thinking mode.temperature=1.0, up to 65,536 output tokens, mean over 4 samples, official graders. Agentic benchmarks (τ²-Bench, BFCL v4, GDPval-S, BrowseComp) have no total output cap and 16,384 output tokens per step unless noted. Responses that run out of token budget, overflow the context or are malformed (for example unclosed thinking) score 0; infrastructure errors are retried.live_multiple samples are excluded because the provider blocked their content.model-00002-of-00004.safetensors 20 GBmodel-00003-of-00004.safetensors 20 GBmodel-00001-of-00004.safetensors 20 GBdspark/model.safetensors 5.5 GBmodel-00004-of-00004.safetensors 4.6 GBsarvamai/sarvam-105b-1711-bf16