A calibrated 4-bit quantization of internlm/Sage-S2.1-Flash
(revision 76f4513db7464485605a002a06a6190de0460574), laid out so that
SparkGLM loads it on two or four NVIDIA DGX Sparks.
| Weights | Format | How |
|---|---|---|
| Routed experts, layers 3 to 44 (42 layers of 288 experts) | NVFP4 | GPTQ, one Hessian per expert from that expert's own inputs, 600K calibration tokens |
| Attention projections of the 34 linear-attention layers, the dense MLP of layers 0 to 2 | MXFP8, 1×32 blocks | Re-rounded against layer inputs, 200K calibration tokens |
Attention projections of the 11 indexed-attention layers (except kv_b_proj, which stays BF16), shared experts |
FP8, 128×128 blocks | Re-rounded against layer inputs, 200K calibration tokens |
lm_head |
MXFP8, 1×32 blocks | Round to nearest |
| Multi-token-prediction (MTP) layer | NVFP4 experts, MXFP8 elsewhere | Round to nearest |
| Vision tower, embeddings, norms, routers | BF16 | Unchanged |
Every routed expert was calibrated from the BF16 weights. No other quantized checkpoint was used as a starting point. The 8-bit matrices keep the formats SparkGLM requires; calibrating their rounding recovered the accuracy that plain rounding of those matrices costs (KL 0.0625 to 0.0531).
Calibration used 2,229 sequences (2.4M tokens available, 600K used for the experts) from public
datasets: reasoning chat, plain chat, tool calling, code, English and multilingual prose, science and
math. The sources and row numbers are listed in receipts/MANIFEST.md.
Measured in PyTorch on 224 held-out documents (239,711 scored positions) that are not in the calibration set. Intervals are 95% bootstrap intervals over documents.
| KL | Top-1 agreement | NLL increase | |
|---|---|---|---|
| This checkpoint | 0.0531 [0.0441, 0.0631] | 94.50% [94.10, 94.88] | +0.64% |
Bizuayeu/Sage-S2.1-Flash-Mixed-NVFP4 (32063d6) |
0.0676 [0.0548, 0.0819] | 93.89% | +0.66% |
Paired over the same documents, this checkpoint's KL is lower by 0.0145 [0.0100, 0.0197] and its top-1
agreement higher by 0.61 points [0.45, 0.78]; its KL is lower on 157 of the 224 documents. The other
checkpoint keeps the linear-attention projections and lm_head in BF16, which SparkGLM does not
load, so the two are not the same size or layout. Its own model card reports different numbers
because it uses a different document set.
By category (32 documents each), KL and top-1 agreement:
| Category | This checkpoint | Bizuayeu Mixed-NVFP4 |
|---|---|---|
| Reasoning chat | 0.0573, 95.35% | 0.0679, 94.35% |
| Code | 0.0276, 96.84% | 0.0263, 96.92% |
| Math | 0.0199, 96.35% | 0.0234, 96.11% |
| English prose | 0.0424, 94.25% | 0.0415, 94.40% |
| Japanese prose | 0.0265, 94.04% | 0.0280, 93.64% |
| Science | 0.0159, 95.18% | 0.0163, 95.11% |
| Tool calling | 0.2772, 85.93% | 0.4176, 81.53% |
Most of the overall gain is in tool calling. On code and English prose the two are level, with this checkpoint slightly behind.
Checked with two Sparks (tensor parallel 2) in Spark mode. Decode speed and the qualification suite are from SparkGLM 0.3.6, installed through LLooM on 2026-11-18; the image, video and other probes below were run on 0.3.5:
| Sage S2.1 Flash NVFP4 | GLM-5.3-Flash FP4-Local, same pair | |
|---|---|---|
| Prose decode | 38.6 tok/s | 37.7 tok/s |
| Code decode | 67.9 tok/s | 66.1 tok/s |
| Structured output decode | 90.0 tok/s | 88.8 tok/s |
| Prompt cache pool | 2.22M tokens | 2.38M tokens |
Speeds are medians of three runs. The pool sizes are from consecutive starts and vary by a few percent from start to start.
Forced tool choice needs SparkGLM 0.3.6. On SparkGLM 0.3.5, a request that forces a tool
(tool_choice naming a tool, required, or any) can come back with the call written as text
instead of as a tool call. The cause is how the engine masked the first token of a forced call: it allowed
the tag's first character as plain text. Requests with automatic tool choice are not affected.
The fix (Enntity/atlas#33) is in
SparkGLM 0.3.6, the current release,
which is that fix on top of 0.3.5. The
0.3.6 results
describe the fault and the checks.
Use Spark mode. Sage reasons before answering even when a request switches thinking off. Spark
mode caps that reasoning; Strict mode, the default of SparkGLM's start.sh, does not. In Strict mode
on the engine commit that 0.3.6 ships, the suite passes 97 of 109: the 12 failures are three prompts that ask the
model to repeat tool-call markup as plain text (each sent four ways), where the longer reasoning
leads it to a different answer.
There are two ways to serve it:
Through LLooM, which is how the numbers above were
measured. Its SparkGLM recipe installs this checkpoint in Spark mode on the 0.3.6 image, pinned at
revision 6967e495 of this repository (the same weight files as the current revision):
lloom setup --recipe linux-nvidia-dgx-spark-glm53-atlas --model sage-s21-flash --additive --apply --yes
The check started from a local copy of the weights, which the Hugging Face tool verified against that revision; a first download of all 179 GB through setup was not run.
With SparkGLM's start.sh: check out v0.3.6 or later, set
CHECKPOINT_REPO=Enntity/Sage-S2.1-Flash-NVFP4 and CHECKPOINT_REVISION= the commit to pin in
.env, and start with ./start.sh --mode spark. This path has not been run end to end with this
checkpoint.
The drafter SparkGLM uses for speculative decoding was trained on GLM-5.3-Flash, not on Sage.
--memory shared
and --streams 8 options.<tool_call> inside
message content as ordinary text. SparkGLM has its own handling for literal markup; its tests for
that pass with this checkpoint, but it is not the original code path.SHA256SUMS lists the weight, tokenizer and configuration files. deployment_guide.md is the original repository's guide, unchanged.
receipts/ holds the evaluation outputs behind the tables above, the script that computes the paired
comparison, the list of calibration and evaluation sources, and the SparkGLM suite and speed results.
model-rest-00001.safetensors 4.3 GBmodel-rest-00002.safetensors 4.3 GBmodel-rest-00003.safetensors 4.3 GBmodel-experts-L15.safetensors 3.9 GBmodel-experts-L12.safetensors 3.9 GBmodel-experts-L16.safetensors 3.9 GBmodel-experts-L27.safetensors 3.9 GBmodel-experts-L14.safetensors 3.9 GBmodel-experts-L25.safetensors 3.9 GBmodel-experts-L19.safetensors 3.9 GBmodel-experts-L26.safetensors 3.9 GBmodel-experts-L17.safetensors 3.9 GBmodel-experts-L34.safetensors 3.9 GBmodel-experts-L28.safetensors 3.9 GBmodel-experts-L31.safetensors 3.9 GBmodel-experts-L11.safetensors 3.9 GBmodel-experts-L30.safetensors 3.9 GBmodel-experts-L13.safetensors 3.9 GBmodel-experts-L44.safetensors 3.9 GBmodel-experts-L36.safetensors 3.9 GBmodel-experts-L33.safetensors 3.9 GBmodel-experts-L18.safetensors 3.9 GBmodel-experts-L09.safetensors 3.9 GBmodel-experts-L35.safetensors 3.9 GBmodel-experts-L37.safetensors 3.9 GBmodel-experts-L08.safetensors 3.9 GBmodel-experts-L07.safetensors 3.9 GBmodel-experts-L05.safetensors 3.9 GBmodel-experts-L32.safetensors 3.9 GBmodel-experts-L04.safetensors 3.9 GBmodel-experts-L29.safetensors 3.9 GBmodel-experts-L10.safetensors 3.9 GBmodel-experts-L06.safetensors 3.9 GBmodel-experts-L38.safetensors 3.9 GBmodel-experts-L20.safetensors 3.9 GBmodel-experts-L39.safetensors 3.9 GBmodel-experts-L24.safetensors 3.9 GBmodel-experts-L21.safetensors 3.9 GBmodel-experts-L22.safetensors 3.9 GBmodel-experts-L40.safetensors 3.9 GBmodel-experts-L23.safetensors 3.9 GBmodel-experts-L03.safetensors 3.9 GBmodel-experts-L43.safetensors 3.9 GBmodel-experts-L41.safetensors 3.9 GBmodel-experts-L42.safetensors 3.9 GBmodel-rest-00004.safetensors 2.7 GBinternlm/Sage-S2.1-Flash