An experimental 2-bit mixed-precision quantization of the Sage S2.1 Flash scientific multimodal model. These are quantized stock weights — NO behavioral modification.
EXPERIMENTAL: This is a 2-bit experimental artifact. Quality can differ materially from the bfloat16 original. See Limitations.
| Item | Status |
|---|---|
| Language model | 2-bit mixed-precision quantization validated for serving |
| Vision tower, projectors, MTP | Copied unquantized from the parent release |
| Modality | Text; image and scientific inputs follow the parent model's tooling |
| Quantization method | Offline weight-only conversion; no calibration or fine-tuning |
| Comprehensive evaluation | Not yet published |
| Behavioral modification | None |
This repository is an experiment in extreme low-bit conversion of a large multimodal mixture-of-experts model, in the style of NVIDIA's 2-bit Nemotron releases. The goal is to fit a 320B-A18B model's weights in roughly 103 GiB — small enough for a single 128 GB unified-memory GPU — while keeping the components most sensitive to quantization error at high precision. No behavioral modification was applied: the weights are quantized stock weights from the parent release.
| Property | Value |
|---|---|
| Architecture | GLM-5.3-Flash-architecture scientific multimodal MoE (glm5_next) |
| Parent parameters | 320B-A18B (45 layers, 288 routed experts, 8 active per token) |
| Language weight format | Mixed-precision: IQ2_XXS routed experts; Q8_0 shared and dense MLP; BF16 attention (MLA), linear attention (KDA), embeddings, LM head, vision tower, MTP layer |
| Model artifact size | 110,911,082,224 bytes (approximately 103.29 GiB) across 120 safetensors shards |
| Upstream context claim | 1,048,576 tokens |
| Validated context in this release | 32,768 tokens |
| Validated concurrency | 4 parallel slots (short-input smoke) |
| License | MIT, inherited from the parent release |
The upstream context specification is not a claim that this derivative was validated at the full 1M window. The upstream benchmark scores below belong to the parent bfloat16 release and do not automatically transfer to this quantized artifact.
internlm/Sage-S2.1-Flash (BF16, MIT)
└── SAGE S2.1 FLASH — EXPERIMENTAL 2-BIT
| Stage | Provenance |
|---|---|
| Foundation | internlm/Sage-S2.1-Flash per the parent's own metadata |
| Immediate parent | internlm/Sage-S2.1-Flash |
| Parent revision | 76f4513db7464485605a002a06a6190de0460574 |
| Conversion | Offline weight-only 2-bit conversion of the parent release's safetensors |
| Conversion tooling | NVIDIA TensorRT Model Optimizer (modelopt) 0.48.0, ggml-style block formats via the same container format as NVIDIA's 2-bit Nemotron releases |
| Derivative stage | Quantization only. No fine-tuning, no calibration, no behavioral modification |
| Runtime lineage | Validated on vLLM 0.27.1 with b12x kernels (container format shared with NVIDIA's 2-bit Nemotron releases) |
The conversion retains the parent's tokenizer, processor code, chat template, projectors, vendor deployment documentation, and original card (UPSTREAM-README.md).
Verification: the shipped shards match the conversion receipts byte-for-byte (all 120 of 120 shards re-verified by SHA-256 against the build journal before upload).
This artifact was produced by an offline, weight-only mixed-precision quantization of the parent bfloat16 release. The process: select per-module quantization formats by surface sensitivity, pack the routed expert weights into 2.0625-bit IQ2_XXS blocks, pack the shared-expert and dense-MLP weights into 8.5-bit Q8_0 blocks, and copy every other tensor verbatim from the source. No calibration data, fine-tuning, activation quantization, or weight editing was involved.
| Surface | Layers | Format | Bytes out (approx.) |
|---|---|---|---|
| Routed experts (gate/up/down) | 42 × 288 | IQ2_XXS (2.0625 bpw, ggml 256-in/66-out blocks) | 78.45 GB |
| Shared experts + dense MLP | 42 shared + 3 dense | Q8_0 (8.5 bpw, ggml 32-in/34-out blocks) | 1.84 GB |
| MLA attention, KDA linear attention | 12 + 34 layers | BF16, unquantized | 12.22 GB |
| Embeddings, LM head | — | BF16, unquantized | 2.53 GB |
| Vision tower, 9 scientific projectors | — | BF16, unquantized | ~1.0 GB |
| MTP layer 45 (speculative-decode weights) | 1 | BF16, unquantized | ~14.8 GB |
Pack fidelity at conversion time (tensor-level cosine similarity vs bfloat16, measured on sample tensors): IQ2_XXS ≈ 0.9446, Q8_0 ≈ 0.999985. These are conversion-fidelity observations, not end-to-end quality benchmarks.
Boundary artifacts of the 120-shard set (verify after download):
| Artifact | Bytes | SHA-256 |
|---|---|---|
model-00001-of-00120.safetensors |
3,391,554,680 | b737792a69e5aff14ef1ba747cc5319a27ffc80c60677b7adc300f2ebfa4fe52 |
model-00120-of-00120.safetensors |
2,279,833,216 | 7a6f2330774882be8f2bf0fe7987910e57ec786ed60f780f83ffab06604d9fcb |
config.json |
7,463,952 | c54a0f8b8a52261eca06cf76efaf4e513c6fefa42e2586d53ecf2235edee4334 |
model.safetensors.index.json |
4,005,617 | 967f8031beb605c713f7c47a5df71a3f36f6b7bef23faa6649f55aa03f6d5282 |
tokenizer.json |
20,217,442 | 19e773648cb4e65de8660ea6365e10acca112d42a854923df93db4a6f333a82d |
chat_template.jinja |
10,583 | 66a4b6d9bc6ee2eb11fe62472ae237adc6f8428fad4857138dd9806bbac2740e |
Full per-file SHA-256 manifest: MANIFEST-SHA256SUMS.txt.
| Benchmark | Parent (BF16) |
|---|---|
| Terminal-Bench 2.1 | 87.9 |
| DeepSWE | 67.9 |
| SWE-bench Pro | 60.6 |
| OfficeQA Pro | 72.2 |
| MLE-bench Lite | 90.3 |
| DeepResearch Bench II | 54.2 |
The repository retains the parent's chat template, tokenizer, and processor code. Use role-structured chat messages. The parent's tool-calling formats (glm47 parsers) are supported by the validated runtime.
The exact validated single-GPU serving path is documented in DEPLOYMENT/README.md.
The kit contains a sanitized launcher, environment template, and smoke test. It reproduces the validated runtime flags without publishing a live endpoint, internal hostname, user account, absolute path, credential, or model-modification recipe.
vllm_patch/ targets vLLM 0.33.0; the validated runtime here is vLLM 0.27.1 (see deployment kit notes).Released under the MIT license, subject to the parent model's license and notices. Review the upstream model and dataset terms before redistribution or deployment.
This experimental artifact is provided without warranties. Users are responsible for validating fitness, accuracy, security, and legal compliance for their use case.
For reproducible artifact issues, use the repository's Discussions or issue-tracking channel without posting secrets or private infrastructure details.
model-00001-of-00120.safetensors 3.4 GBmodel-00011-of-00120.safetensors 3.1 GBmodel-00120-of-00120.safetensors 2.3 GBmodel-00002-of-00120.safetensors 2.0 GBmodel-00005-of-00120.safetensors 2.0 GBmodel-00008-of-00120.safetensors 1.9 GBmodel-00010-of-00120.safetensors 1.8 GBmodel-00007-of-00120.safetensors 1.8 GBmodel-00004-of-00120.safetensors 1.7 GBmodel-00009-of-00120.safetensors 1.7 GBmodel-00006-of-00120.safetensors 1.7 GBmodel-00003-of-00120.safetensors 1.7 GBmodel-00076-of-00120.safetensors 1.5 GBmodel-00098-of-00120.safetensors 1.5 GBmodel-00069-of-00120.safetensors 1.5 GBmodel-00091-of-00120.safetensors 1.5 GBmodel-00062-of-00120.safetensors 1.5 GBmodel-00047-of-00120.safetensors 1.5 GBmodel-00113-of-00120.safetensors 1.5 GBmodel-00084-of-00120.safetensors 1.5 GBmodel-00055-of-00120.safetensors 1.5 GBmodel-00018-of-00120.safetensors 1.5 GBmodel-00040-of-00120.safetensors 1.5 GBmodel-00033-of-00120.safetensors 1.5 GBmodel-00026-of-00120.safetensors 1.5 GBmodel-00105-of-00120.safetensors 1.4 GBmodel-00106-of-00120.safetensors 751 MBmodel-00050-of-00120.safetensors 692 MBmodel-00051-of-00120.safetensors 692 MBmodel-00052-of-00120.safetensors 692 MBmodel-00053-of-00120.safetensors 692 MBmodel-00054-of-00120.safetensors 692 MBmodel-00057-of-00120.safetensors 692 MBmodel-00058-of-00120.safetensors 692 MBmodel-00059-of-00120.safetensors 692 MBmodel-00060-of-00120.safetensors 692 MBmodel-00061-of-00120.safetensors 692 MBmodel-00064-of-00120.safetensors 692 MBmodel-00065-of-00120.safetensors 692 MBmodel-00066-of-00120.safetensors 692 MBmodel-00067-of-00120.safetensors 692 MBmodel-00068-of-00120.safetensors 692 MBmodel-00071-of-00120.safetensors 692 MBmodel-00072-of-00120.safetensors 692 MBmodel-00073-of-00120.safetensors 692 MBmodel-00074-of-00120.safetensors 692 MBmodel-00075-of-00120.safetensors 692 MBmodel-00079-of-00120.safetensors 692 MBmodel-00080-of-00120.safetensors 692 MBmodel-00081-of-00120.safetensors 692 MBmodel-00082-of-00120.safetensors 692 MBmodel-00083-of-00120.safetensors 692 MBmodel-00086-of-00120.safetensors 692 MBmodel-00087-of-00120.safetensors 692 MBmodel-00088-of-00120.safetensors 692 MBmodel-00089-of-00120.safetensors 692 MBmodel-00090-of-00120.safetensors 692 MBmodel-00093-of-00120.safetensors 692 MBmodel-00094-of-00120.safetensors 692 MBmodel-00095-of-00120.safetensors 692 MBmodel-00096-of-00120.safetensors 692 MBmodel-00097-of-00120.safetensors 692 MBmodel-00100-of-00120.safetensors 692 MBmodel-00101-of-00120.safetensors 692 MBmodel-00102-of-00120.safetensors 692 MBmodel-00103-of-00120.safetensors 692 MBmodel-00104-of-00120.safetensors 692 MBmodel-00108-of-00120.safetensors 692 MBmodel-00109-of-00120.safetensors 692 MBmodel-00110-of-00120.safetensors 692 MBmodel-00111-of-00120.safetensors 692 MBmodel-00112-of-00120.safetensors 692 MBmodel-00115-of-00120.safetensors 692 MBmodel-00116-of-00120.safetensors 692 MBmodel-00117-of-00120.safetensors 692 MBmodel-00118-of-00120.safetensors 692 MBmodel-00119-of-00120.safetensors 692 MBmodel-00049-of-00120.safetensors 692 MBmodel-00078-of-00120.safetensors 692 MBmodel-00107-of-00120.safetensors 692 MBmodel-00056-of-00120.safetensors 692 MBmodel-00085-of-00120.safetensors 692 MBmodel-00114-of-00120.safetensors 692 MBmodel-00046-of-00120.safetensors 692 MBmodel-00042-of-00120.safetensors 692 MBmodel-00043-of-00120.safetensors 692 MBmodel-00044-of-00120.safetensors 692 MBmodel-00045-of-00120.safetensors 692 MBmodel-00063-of-00120.safetensors 692 MBmodel-00092-of-00120.safetensors 692 MBmodel-00070-of-00120.safetensors 692 MBmodel-00099-of-00120.safetensors 692 MBmodel-00013-of-00120.safetensors 692 MBmodel-00014-of-00120.safetensors 692 MBmodel-00015-of-00120.safetensors 692 MBmodel-00016-of-00120.safetensors 692 MBmodel-00017-of-00120.safetensors 692 MBmodel-00021-of-00120.safetensors 692 MBmodel-00022-of-00120.safetensors 692 MBmodel-00023-of-00120.safetensors 692 MBmodel-00024-of-00120.safetensors 692 MBmodel-00025-of-00120.safetensors 692 MBmodel-00028-of-00120.safetensors 692 MBmodel-00029-of-00120.safetensors 692 MBmodel-00030-of-00120.safetensors 692 MBmodel-00031-of-00120.safetensors 692 MBmodel-00032-of-00120.safetensors 692 MBmodel-00035-of-00120.safetensors 692 MBmodel-00036-of-00120.safetensors 692 MBmodel-00037-of-00120.safetensors 692 MBmodel-00038-of-00120.safetensors 692 MBmodel-00039-of-00120.safetensors 692 MBmodel-00048-of-00120.safetensors 692 MBmodel-00077-of-00120.safetensors 692 MBmodel-00020-of-00120.safetensors 692 MBmodel-00027-of-00120.safetensors 692 MBmodel-00041-of-00120.safetensors 692 MBmodel-00034-of-00120.safetensors 692 MBmodel-00012-of-00120.safetensors 692 MBmodel-00019-of-00120.safetensors 692 MBprojectors/sage-s2.1-flash/protein-seq/mpbind/projector.safetensors 28 MBprojectors/sage-s2.1-flash/rna/ncrna/projector.safetensors 25 MBprojectors/sage-s2.1-flash/ecg/phu01/projector.safetensors 20 MBprojectors/sage-s2.1-flash/eeg/neu06/projector.safetensors 19 MBprojectors/sage-s2.1-flash/xrd/simxrd/projector.safetensors 19 MBprojectors/sage-s2.1-flash/protein-struct/psr/projector.safetensors 19 MBprojectors/sage-s2.1-flash/cif/jarvis/projector.safetensors 17 MBprojectors/sage-s2.1-flash/spectra/qm9s/projector.safetensors 17 MBprojectors/sage-s2.1-flash/dna/splice/projector.safetensors 4 MBinternlm/Sage-S2.1-Flash