An untouched EXL3 quantization of internlm/Sage-S2.1-Flash. No abliteration or fine-tuning was applied. Quantization changes weight precision.
76f4513db7464485605a002a06a6190de0460574.This is mixed precision, not uniform 4.25-bit storage. Nine routed-expert blocks use 5 bits (physical blocks 3, 4, 5, and 39-44); other routed blocks use 4 bits. Attention/shared experts use 6 bits, except kv_a_proj_with_mqa in blocks 3 and 43, which uses 7 bits; dense FFNs use 5 bits. Embeddings, output head, and vision use retained 16-bit storage. Scientific projectors are copied unchanged; norms, routers, mHC and other retained tensors have their exact serialized dtypes listed in the tensor inventory. See PRECISION-ALLOCATION.csv for every quantized linear, LAYER-PRECISION.csv for every physical block, and TENSOR-INVENTORY.csv for every serialized tensor, including retained weights, packed dtype, shape, and exact byte size. quantization_config.json records serialized per-tensor storage and precision-map.json preserves the frozen allocation (map 8 applies to this artifact). Allocated main bpw excludes retained components and MTP, so it is not a whole-repository byte rate.
Calibration uses 128 sequences of 4,096 tokens: 32 each for math, science, code, and multi-turn tools. Main quantization is performed once from original BF16. Integrated MTP calibration uses preceding main hidden states paired with the next input token. The source revision, calibration hash, precision checks, and artifact hashes are recorded in sage-provenance.json and artifact-files.json. Raw private calibration traces, benchmark questions, and generations are not distributed.
Use the serving source, reference profile, and serving guide. The profile uses a three-GPU direct P2P split, a batched indexer, and the measured settings for lower CPU use. The archive contains source code.
Download the complete snapshot with:
hf download SourceHat-Labs/Sage-S2.1-Flash-EXL3-4.25bpw --local-dir ./Sage-S2.1-Flash-EXL3-4.25bpw
The pinned upstream README declares MIT for code and weights. Its original text is preserved in UPSTREAM-README.md; see UPSTREAM-LICENSE-NOTICE.md. The bundled EXL3 source retains its original license and copyright notices.
All values below are weight bits. Blocks 0-44 are the main decoder; block 45 is MTP. Precision is assigned per projection matrix and applies across its attention heads. Packed tensor dtype is reported separately in TENSOR-INVENTORY.csv.
| Physical block | Attention | Attention/input projection bits | Dense/routed FFN bits | Shared FFN bits |
|---|---|---|---|---|
| 0 | KDA | o=6; qkv=6 | down=5; gate=5; up=5 | - |
| 1 | KDA | o=6; qkv=6 | down=5; gate=5; up=5 | - |
| 2 | KDA | o=6; qkv=6 | down=5; gate=5; up=5 | - |
| 3 | DSA | indexer.wq_b=6; kv_a_mqa=7; o=6; q_a=6; q_b=6 | down=5 (288 experts); gate=5 (288 experts); up=5 (288 experts) | down=6; gate=6; up=6 |
| 4 | KDA | o=6; qkv=6 | down=5 (288 experts); gate=5 (288 experts); up=5 (288 experts) | down=6; gate=6; up=6 |
| 5 | KDA | o=6; qkv=6 | down=5 (288 experts); gate=5 (288 experts); up=5 (288 experts) | down=6; gate=6; up=6 |
| 6 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 7 | DSA | indexer.wq_b=6; kv_a_mqa=6; o=6; q_a=6; q_b=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 8 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 9 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 10 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 11 | DSA | indexer.wq_b=6; kv_a_mqa=6; o=6; q_a=6; q_b=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 12 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 13 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 14 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 15 | DSA | indexer.wq_b=6; kv_a_mqa=6; o=6; q_a=6; q_b=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 16 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 17 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 18 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 19 | DSA | indexer.wq_b=6; kv_a_mqa=6; o=6; q_a=6; q_b=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 20 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 21 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 22 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 23 | DSA | indexer.wq_b=6; kv_a_mqa=6; o=6; q_a=6; q_b=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 24 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 25 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 26 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 27 | DSA | indexer.wq_b=6; kv_a_mqa=6; o=6; q_a=6; q_b=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 28 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 29 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 30 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 31 | DSA | indexer.wq_b=6; kv_a_mqa=6; o=6; q_a=6; q_b=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 32 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 33 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 34 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 35 | DSA | indexer.wq_b=6; kv_a_mqa=6; o=6; q_a=6; q_b=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 36 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 37 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 38 | KDA | o=6; qkv=6 | down=4 (288 experts); gate=4 (288 experts); up=4 (288 experts) | down=6; gate=6; up=6 |
| 39 | DSA | indexer.wq_b=6; kv_a_mqa=6; o=6; q_a=6; q_b=6 | down=5 (288 experts); gate=5 (288 experts); up=5 (288 experts) | down=6; gate=6; up=6 |
| 40 | KDA | o=6; qkv=6 | down=5 (288 experts); gate=5 (288 experts); up=5 (288 experts) | down=6; gate=6; up=6 |
| 41 | KDA | o=6; qkv=6 | down=5 (288 experts); gate=5 (288 experts); up=5 (288 experts) | down=6; gate=6; up=6 |
| 42 | KDA | o=6; qkv=6 | down=5 (288 experts); gate=5 (288 experts); up=5 (288 experts) | down=6; gate=6; up=6 |
| 43 | DSA | indexer.wq_b=6; kv_a_mqa=7; o=6; q_a=6; q_b=6 | down=5 (288 experts); gate=5 (288 experts); up=5 (288 experts) | down=6; gate=6; up=6 |
| 44 | KDA | o=6; qkv=6 | down=5 (288 experts); gate=5 (288 experts); up=5 (288 experts) | down=6; gate=6; up=6 |
| 45 | MTP | eh_proj=8; indexer.wq_b=8; kv_a_mqa=8; o=8; q_a=8; q_b=8 | down=8 (288 experts); gate=8 (288 experts); up=8 (288 experts) | down=8; gate=8; up=8 |
| Component | Bytes | GiB |
|---|---|---|
| Attention/KDA projections | 4,950,793,196 | 4.610785 |
| Dense FFNs | 283,410,468 | 0.263947 |
| Embedding | 1,268,776,960 | 1.181641 |
| MTP | 7,496,139,816 | 6.981324 |
| Norms, routers, mHC and other tensors | 241,451,768 | 0.224869 |
| Output head | 1,268,776,960 | 1.181641 |
| Routed experts | 160,802,682,624 | 149.759168 |
| Scientific projectors | 168,415,636 | 0.156849 |
| Shared experts | 794,272,248 | 0.739724 |
| Vision | 1,127,254,016 | 1.049837 |
QUANTIZATION-SUMMARY.json includes the allocation and retained embedding/head/vision tensor details.
model-00026-of-00026.safetensors 8.6 GBmodel-00003-of-00026.safetensors 8.4 GBmodel-00006-of-00026.safetensors 7.5 GBmodel-00008-of-00026.safetensors 7.5 GBmodel-00010-of-00026.safetensors 7.5 GBmodel-00012-of-00026.safetensors 7.5 GBmodel-00014-of-00026.safetensors 7.5 GBmodel-00016-of-00026.safetensors 7.5 GBmodel-00018-of-00026.safetensors 7.5 GBmodel-00004-of-00026.safetensors 7.5 GBmodel-00007-of-00026.safetensors 7.5 GBmodel-00009-of-00026.safetensors 7.5 GBmodel-00011-of-00026.safetensors 7.5 GBmodel-00013-of-00026.safetensors 7.5 GBmodel-00015-of-00026.safetensors 7.5 GBmodel-00017-of-00026.safetensors 7.5 GBmodel-00019-of-00026.safetensors 7.5 GBmodel-00005-of-00026.safetensors 7.5 GBmodel-00001-of-00026.safetensors 6.6 GBmodel-00025-of-00026.safetensors 6.0 GBmodel-00024-of-00026.safetensors 4.7 GBmodel-00020-of-00026.safetensors 4.7 GBmodel-00021-of-00026.safetensors 4.7 GBmodel-00022-of-00026.safetensors 4.7 GBmodel-00023-of-00026.safetensors 4.7 GBmodel-00002-of-00026.safetensors 4.7 GBprojectors/sage-s2.1-flash/protein-seq/mpbind/projector.safetensors 28 MBprojectors/sage-s2.1-flash/rna/ncrna/projector.safetensors 25 MBprojectors/sage-s2.1-flash/ecg/phu01/projector.safetensors 20 MBprojectors/sage-s2.1-flash/eeg/neu06/projector.safetensors 19 MBprojectors/sage-s2.1-flash/xrd/simxrd/projector.safetensors 19 MBprojectors/sage-s2.1-flash/protein-struct/psr/projector.safetensors 19 MBprojectors/sage-s2.1-flash/cif/jarvis/projector.safetensors 17 MBprojectors/sage-s2.1-flash/spectra/qm9s/projector.safetensors 17 MBprojectors/sage-s2.1-flash/dna/splice/projector.safetensors 4 MBinternlm/Sage-S2.1-Flash