thinkingmachines/Inkling-2-Small-MXFP8 with its routed experts quantized to 4-bit EXL3, plus three choices of dense core in one repository: the original BF16 core and compact EXL3 cores at 8-bit and 6-bit. You select the core at load time; the routed experts are shared by all three views and stored once.
Built for cuteafd. Current serving status and measured layouts are listed below.
A mixture-of-experts model has two very different kinds of weights:
original view, including EXL3 scales and multipliers), but each token touches only a few of them. These expert weights are shared across views.A multi-core (MC) checkpoint ships the experts once and the core several ways:
| Core view | Core format | Total size | Use it when |
|---|---|---|---|
original (default) |
source-native BF16 core | 290.3 GiB | you want the source core without an additional core quantization |
k8-compact |
EXL3 MCG 8-bit core | 283.5 GiB | you want a smaller core with a supported K8 runtime |
k6-compact |
EXL3 MCG 6-bit core | 281.8 GiB | you want the smallest stored core |
Sizes are tensor payloads from each view index's metadata.total_size, not runtime GPU requirements or the disk space of all alternate cores combined. Native tensors outside the selected core remain shared.
Each view has its own index, config and quantization metadata (model.safetensors.index.json / config.json / quantize_config.json for original, and the .k8. / .k6. siblings). cuteafd-mc.json lists the views. In cuteafd, CORE_VIEW=original|k8-compact|k6-compact selects one; the default is original. See the MC format.
thinkingmachines/Inkling-2-Small-MXFP8@3febc62259ee92ea36e2fb6e8303371c4eb3922c.original view has the BF16 target core and BF16 drafter. Remaining native drafter tensors are shared; there is no separate copy of the K4 routed experts for the drafter.main), 5d93c9de5c2caf151cbe26cbb65b5f4fbee5dc98 (fork in package provenance; no gptqmodel_commit field is published).gptqmodel/utils/inkling2_core.py at the recorded fork commit is not present in upstream GPTQModel. Production orchestration also lives in cuteafd's quantization/inkling2/. K8 uses the fork's K1-K8 projection-adapter widening (tpurtell/GPTQModel#5); upstream shared GLM projection validation still accepts only K4/K6. This complete recipe is therefore not upstream.| Setting | Value |
|---|---|
| Codebook | EXL3 MCG, multiplier 0xCBAC1FED |
| Bits | routed experts K4; selected target/drafter core K8 or K6 |
| Seed / Hessian regularization (sigma) | 787 / 0.025 |
| Output scales | automatic |
| Covariance | FP32 products over at most 128 rows, FP64 outer sum, FP32 storage |
| Activation stream | K8 compact core and K4 experts; K6 reuses captured Hessians |
| Drafter calibration anchor | last complete block per untruncated prompt, using K8 target features |
| Calibration | own-text panel; dataset identity, revision, split and counts not recorded |
Provenance and validation receipts are in package-provenance.json and package-validation.json; per-module formats and bit widths are in quantize_config*.json:tensor_storage.
Use a cuteafd build with MC core-view support; see the Inkling 2 Small family page and the README results tables.
Inkling 2 Small is still being qualified (#1930). The family page describes an initial text-only path with one RTX and 2-8 Sparks, but startup, native cold-prefill/cache exactness, full-model fidelity and served smoke remain pending. This checkpoint has no measured row in the current benchmark matrix. The stored vision tower and drafter do not by themselves establish runtime support or qualification.
Other runtimes need EXL3 MCG support at the expert bit widths listed above and support for the selected core view and the family's preserved native components. The original view avoids additional target-core quantization; it does not remove the routed experts' EXL3 requirement.
Apache-2.0, as declared by this repository. See the included license assets; this repository changes the weight encoding.
shared-layer-011-0001.safetensors 2.0 GBshared-layer-017-0001.safetensors 2.0 GBshared-layer-023-0001.safetensors 2.0 GBshared-layer-029-0001.safetensors 2.0 GBshared-layer-035-0001.safetensors 2.0 GBshared-layer-041-0001.safetensors 2.0 GBshared-layer-047-0001.safetensors 2.0 GBshared-layer-005-0001.safetensors 2.0 GBshared-layer-010-0001.safetensors 2.0 GBshared-layer-012-0001.safetensors 2.0 GBshared-layer-013-0001.safetensors 2.0 GBshared-layer-014-0001.safetensors 2.0 GBshared-layer-015-0001.safetensors 2.0 GBshared-layer-016-0001.safetensors 2.0 GBshared-layer-018-0001.safetensors 2.0 GBshared-layer-019-0001.safetensors 2.0 GBshared-layer-020-0001.safetensors 2.0 GBshared-layer-021-0001.safetensors 2.0 GBshared-layer-022-0001.safetensors 2.0 GBshared-layer-024-0001.safetensors 2.0 GBshared-layer-025-0001.safetensors 2.0 GBshared-layer-026-0001.safetensors 2.0 GBshared-layer-027-0001.safetensors 2.0 GBshared-layer-028-0001.safetensors 2.0 GBshared-layer-030-0001.safetensors 2.0 GBshared-layer-031-0001.safetensors 2.0 GBshared-layer-032-0001.safetensors 2.0 GBshared-layer-033-0001.safetensors 2.0 GBshared-layer-034-0001.safetensors 2.0 GBshared-layer-036-0001.safetensors 2.0 GBshared-layer-037-0001.safetensors 2.0 GBshared-layer-038-0001.safetensors 2.0 GBshared-layer-039-0001.safetensors 2.0 GBshared-layer-040-0001.safetensors 2.0 GBshared-layer-042-0001.safetensors 2.0 GBshared-layer-043-0001.safetensors 2.0 GBshared-layer-044-0001.safetensors 2.0 GBshared-layer-045-0001.safetensors 2.0 GBshared-layer-046-0001.safetensors 2.0 GBshared-layer-000-0001.safetensors 2.0 GBshared-layer-001-0001.safetensors 2.0 GBshared-layer-002-0001.safetensors 2.0 GBshared-layer-003-0001.safetensors 2.0 GBshared-layer-004-0001.safetensors 2.0 GBshared-layer-006-0001.safetensors 2.0 GBshared-layer-007-0001.safetensors 2.0 GBshared-layer-008-0001.safetensors 2.0 GBshared-layer-009-0001.safetensors 2.0 GBshared-layer-010-0002.safetensors 2.0 GBshared-layer-011-0002.safetensors 2.0 GBshared-layer-012-0002.safetensors 2.0 GBshared-layer-013-0002.safetensors 2.0 GBshared-layer-014-0002.safetensors 2.0 GBshared-layer-015-0002.safetensors 2.0 GBshared-layer-016-0002.safetensors 2.0 GBshared-layer-017-0002.safetensors 2.0 GBshared-layer-018-0002.safetensors 2.0 GBshared-layer-019-0002.safetensors 2.0 GBshared-layer-020-0002.safetensors 2.0 GBshared-layer-021-0002.safetensors 2.0 GBshared-layer-022-0002.safetensors 2.0 GBshared-layer-023-0002.safetensors 2.0 GBshared-layer-024-0002.safetensors 2.0 GBshared-layer-025-0002.safetensors 2.0 GBshared-layer-026-0002.safetensors 2.0 GBshared-layer-027-0002.safetensors 2.0 GBshared-layer-028-0002.safetensors 2.0 GBshared-layer-029-0002.safetensors 2.0 GBshared-layer-030-0002.safetensors 2.0 GBshared-layer-031-0002.safetensors 2.0 GBshared-layer-032-0002.safetensors 2.0 GBshared-layer-033-0002.safetensors 2.0 GBshared-layer-034-0002.safetensors 2.0 GBshared-layer-035-0002.safetensors 2.0 GBshared-layer-036-0002.safetensors 2.0 GBshared-layer-037-0002.safetensors 2.0 GBshared-layer-038-0002.safetensors 2.0 GBshared-layer-039-0002.safetensors 2.0 GBshared-layer-040-0002.safetensors 2.0 GBshared-layer-041-0002.safetensors 2.0 GBshared-layer-042-0002.safetensors 2.0 GBshared-layer-043-0002.safetensors 2.0 GBshared-layer-044-0002.safetensors 2.0 GBshared-layer-045-0002.safetensors 2.0 GBshared-layer-046-0002.safetensors 2.0 GBshared-layer-047-0002.safetensors 2.0 GBshared-layer-010-0003.safetensors 2.0 GBshared-layer-011-0003.safetensors 2.0 GBshared-layer-012-0003.safetensors 2.0 GBshared-layer-013-0003.safetensors 2.0 GBshared-layer-014-0003.safetensors 2.0 GBshared-layer-015-0003.safetensors 2.0 GBshared-layer-016-0003.safetensors 2.0 GBshared-layer-017-0003.safetensors 2.0 GBshared-layer-018-0003.safetensors 2.0 GBshared-layer-019-0003.safetensors 2.0 GBshared-layer-020-0003.safetensors 2.0 GBshared-layer-021-0003.safetensors 2.0 GBshared-layer-022-0003.safetensors 2.0 GBshared-layer-023-0003.safetensors 2.0 GBshared-layer-024-0003.safetensors 2.0 GBshared-layer-025-0003.safetensors 2.0 GBshared-layer-026-0003.safetensors 2.0 GBshared-layer-027-0003.safetensors 2.0 GBshared-layer-028-0003.safetensors 2.0 GBshared-layer-029-0003.safetensors 2.0 GBshared-layer-030-0003.safetensors 2.0 GBshared-layer-031-0003.safetensors 2.0 GBshared-layer-032-0003.safetensors 2.0 GBshared-layer-033-0003.safetensors 2.0 GBshared-layer-034-0003.safetensors 2.0 GBshared-layer-035-0003.safetensors 2.0 GBshared-layer-036-0003.safetensors 2.0 GBshared-layer-037-0003.safetensors 2.0 GBshared-layer-038-0003.safetensors 2.0 GBshared-layer-039-0003.safetensors 2.0 GBshared-layer-040-0003.safetensors 2.0 GBshared-layer-041-0003.safetensors 2.0 GBshared-layer-042-0003.safetensors 2.0 GBshared-layer-043-0003.safetensors 2.0 GBshared-layer-044-0003.safetensors 2.0 GBshared-layer-045-0003.safetensors 2.0 GBshared-layer-046-0003.safetensors 2.0 GBshared-layer-047-0003.safetensors 2.0 GBshared-layer-000-0002.safetensors 2.0 GBshared-layer-001-0002.safetensors 2.0 GBshared-layer-002-0002.safetensors 2.0 GBshared-layer-003-0002.safetensors 2.0 GBshared-layer-004-0002.safetensors 2.0 GBshared-layer-005-0002.safetensors 2.0 GBshared-layer-006-0002.safetensors 2.0 GBshared-layer-007-0002.safetensors 2.0 GBshared-layer-008-0002.safetensors 2.0 GBshared-layer-009-0002.safetensors 2.0 GBshared-layer-000-0003.safetensors 2.0 GBshared-layer-001-0003.safetensors 2.0 GBshared-layer-002-0003.safetensors 2.0 GBshared-layer-003-0003.safetensors 2.0 GBshared-layer-004-0003.safetensors 2.0 GBshared-layer-005-0003.safetensors 2.0 GBshared-layer-006-0003.safetensors 2.0 GBshared-layer-007-0003.safetensors 2.0 GBshared-layer-008-0003.safetensors 2.0 GBshared-layer-009-0003.safetensors 2.0 GBshared-native-0001.safetensors 2.0 GBoriginal-head-0001.safetensors 1.9 GBk8-compact-head-0001.safetensors 927 MBk6-compact-head-0001.safetensors 695 MBoriginal-mtp-000-0001.safetensors 661 MBoriginal-mtp-001-0001.safetensors 661 MBoriginal-mtp-002-0001.safetensors 661 MBoriginal-mtp-003-0001.safetensors 661 MBoriginal-mtp-004-0001.safetensors 661 MBshared-native-0002.safetensors 637 MBk8-compact-mtp-000-0001.safetensors 331 MBk8-compact-mtp-001-0001.safetensors 331 MBk8-compact-mtp-002-0001.safetensors 331 MBk8-compact-mtp-003-0001.safetensors 331 MBk8-compact-mtp-004-0001.safetensors 331 MBk6-compact-mtp-000-0001.safetensors 248 MBk6-compact-mtp-001-0001.safetensors 248 MBk6-compact-mtp-002-0001.safetensors 248 MBk6-compact-mtp-003-0001.safetensors 248 MBk6-compact-mtp-004-0001.safetensors 248 MBoriginal-layer-010-0001.safetensors 196 MBoriginal-layer-011-0001.safetensors 196 MBoriginal-layer-012-0001.safetensors 196 MBoriginal-layer-013-0001.safetensors 196 MBoriginal-layer-014-0001.safetensors 196 MBoriginal-layer-015-0001.safetensors 196 MBoriginal-layer-016-0001.safetensors 196 MBoriginal-layer-017-0001.safetensors 196 MBoriginal-layer-018-0001.safetensors 196 MBoriginal-layer-019-0001.safetensors 196 MBoriginal-layer-020-0001.safetensors 196 MBoriginal-layer-021-0001.safetensors 196 MBoriginal-layer-022-0001.safetensors 196 MBoriginal-layer-023-0001.safetensors 196 MBoriginal-layer-024-0001.safetensors 196 MBoriginal-layer-025-0001.safetensors 196 MBoriginal-layer-026-0001.safetensors 196 MBoriginal-layer-027-0001.safetensors 196 MBoriginal-layer-028-0001.safetensors 196 MBoriginal-layer-029-0001.safetensors 196 MBoriginal-layer-030-0001.safetensors 196 MBoriginal-layer-031-0001.safetensors 196 MBoriginal-layer-032-0001.safetensors 196 MBoriginal-layer-033-0001.safetensors 196 MBoriginal-layer-034-0001.safetensors 196 MBoriginal-layer-035-0001.safetensors 196 MBoriginal-layer-036-0001.safetensors 196 MBoriginal-layer-037-0001.safetensors 196 MBoriginal-layer-038-0001.safetensors 196 MBoriginal-layer-039-0001.safetensors 196 MBoriginal-layer-040-0001.safetensors 196 MBoriginal-layer-041-0001.safetensors 196 MBoriginal-layer-042-0001.safetensors 196 MBoriginal-layer-043-0001.safetensors 196 MBoriginal-layer-044-0001.safetensors 196 MBoriginal-layer-045-0001.safetensors 196 MBoriginal-layer-046-0001.safetensors 196 MBoriginal-layer-047-0001.safetensors 196 MBoriginal-layer-000-0001.safetensors 196 MBoriginal-layer-001-0001.safetensors 196 MBoriginal-layer-002-0001.safetensors 196 MBoriginal-layer-003-0001.safetensors 196 MBoriginal-layer-004-0001.safetensors 196 MBoriginal-layer-005-0001.safetensors 196 MBoriginal-layer-006-0001.safetensors 196 MBoriginal-layer-007-0001.safetensors 196 MBoriginal-layer-008-0001.safetensors 196 MBoriginal-layer-009-0001.safetensors 196 MBshared-layer-011-0004.safetensors 147 MBshared-layer-017-0004.safetensors 147 MBshared-layer-023-0004.safetensors 147 MBshared-layer-029-0004.safetensors 147 MBshared-layer-035-0004.safetensors 147 MBshared-layer-041-0004.safetensors 147 MBshared-layer-047-0004.safetensors 147 MBshared-layer-005-0004.safetensors 147 MBshared-layer-010-0004.safetensors 147 MBshared-layer-012-0004.safetensors 147 MBshared-layer-013-0004.safetensors 147 MBshared-layer-014-0004.safetensors 147 MBshared-layer-015-0004.safetensors 147 MBshared-layer-016-0004.safetensors 147 MBshared-layer-018-0004.safetensors 147 MBshared-layer-019-0004.safetensors 147 MBshared-layer-020-0004.safetensors 147 MBshared-layer-021-0004.safetensors 147 MBshared-layer-022-0004.safetensors 147 MBshared-layer-024-0004.safetensors 147 MBshared-layer-025-0004.safetensors 147 MBshared-layer-026-0004.safetensors 147 MBshared-layer-027-0004.safetensors 147 MBshared-layer-028-0004.safetensors 147 MBshared-layer-030-0004.safetensors 147 MBshared-layer-031-0004.safetensors 147 MBshared-layer-032-0004.safetensors 147 MBshared-layer-033-0004.safetensors 147 MBshared-layer-034-0004.safetensors 147 MBshared-layer-036-0004.safetensors 147 MBshared-layer-037-0004.safetensors 147 MBshared-layer-038-0004.safetensors 147 MBshared-layer-039-0004.safetensors 147 MBshared-layer-040-0004.safetensors 147 MBshared-layer-042-0004.safetensors 147 MBshared-layer-043-0004.safetensors 147 MBshared-layer-044-0004.safetensors 147 MBshared-layer-045-0004.safetensors 147 MBshared-layer-046-0004.safetensors 147 MBshared-layer-000-0004.safetensors 147 MBshared-layer-001-0004.safetensors 147 MBshared-layer-002-0004.safetensors 147 MBshared-layer-003-0004.safetensors 147 MBshared-layer-004-0004.safetensors 147 MBshared-layer-006-0004.safetensors 147 MBshared-layer-007-0004.safetensors 147 MBshared-layer-008-0004.safetensors 147 MBshared-layer-009-0004.safetensors 147 MBk8-compact-layer-010-0001.safetensors 98 MBk8-compact-layer-011-0001.safetensors 98 MBk8-compact-layer-012-0001.safetensors 98 MBk8-compact-layer-013-0001.safetensors 98 MBk8-compact-layer-014-0001.safetensors 98 MBk8-compact-layer-015-0001.safetensors 98 MBk8-compact-layer-016-0001.safetensors 98 MBk8-compact-layer-017-0001.safetensors 98 MBk8-compact-layer-018-0001.safetensors 98 MBk8-compact-layer-019-0001.safetensors 98 MBk8-compact-layer-020-0001.safetensors 98 MBk8-compact-layer-021-0001.safetensors 98 MBk8-compact-layer-022-0001.safetensors 98 MBk8-compact-layer-023-0001.safetensors 98 MBk8-compact-layer-024-0001.safetensors 98 MBk8-compact-layer-025-0001.safetensors 98 MBk8-compact-layer-026-0001.safetensors 98 MBk8-compact-layer-027-0001.safetensors 98 MBk8-compact-layer-028-0001.safetensors 98 MBk8-compact-layer-029-0001.safetensors 98 MBk8-compact-layer-030-0001.safetensors 98 MBk8-compact-layer-031-0001.safetensors 98 MBk8-compact-layer-032-0001.safetensors 98 MBk8-compact-layer-033-0001.safetensors 98 MBk8-compact-layer-034-0001.safetensors 98 MBk8-compact-layer-035-0001.safetensors 98 MBk8-compact-layer-036-0001.safetensors 98 MBk8-compact-layer-037-0001.safetensors 98 MBk8-compact-layer-038-0001.safetensors 98 MBk8-compact-layer-039-0001.safetensors 98 MBk8-compact-layer-040-0001.safetensors 98 MBk8-compact-layer-041-0001.safetensors 98 MBk8-compact-layer-042-0001.safetensors 98 MBk8-compact-layer-043-0001.safetensors 98 MBk8-compact-layer-044-0001.safetensors 98 MBk8-compact-layer-045-0001.safetensors 98 MBk8-compact-layer-046-0001.safetensors 98 MBk8-compact-layer-047-0001.safetensors 98 MBk8-compact-layer-000-0001.safetensors 98 MBk8-compact-layer-001-0001.safetensors 98 MBk8-compact-layer-002-0001.safetensors 98 MBk8-compact-layer-003-0001.safetensors 98 MBk8-compact-layer-004-0001.safetensors 98 MBk8-compact-layer-005-0001.safetensors 98 MBk8-compact-layer-006-0001.safetensors 98 MBk8-compact-layer-007-0001.safetensors 98 MBk8-compact-layer-008-0001.safetensors 98 MBk8-compact-layer-009-0001.safetensors 98 MBk6-compact-layer-010-0001.safetensors 74 MBk6-compact-layer-011-0001.safetensors 74 MBk6-compact-layer-012-0001.safetensors 74 MBk6-compact-layer-013-0001.safetensors 74 MBk6-compact-layer-014-0001.safetensors 74 MBk6-compact-layer-015-0001.safetensors 74 MBk6-compact-layer-016-0001.safetensors 74 MBk6-compact-layer-017-0001.safetensors 74 MBk6-compact-layer-018-0001.safetensors 74 MBk6-compact-layer-019-0001.safetensors 74 MBk6-compact-layer-020-0001.safetensors 74 MBk6-compact-layer-021-0001.safetensors 74 MBk6-compact-layer-022-0001.safetensors 74 MBk6-compact-layer-023-0001.safetensors 74 MBk6-compact-layer-024-0001.safetensors 74 MBk6-compact-layer-025-0001.safetensors 74 MBk6-compact-layer-026-0001.safetensors 74 MBk6-compact-layer-027-0001.safetensors 74 MBk6-compact-layer-028-0001.safetensors 74 MBk6-compact-layer-029-0001.safetensors 74 MBk6-compact-layer-030-0001.safetensors 74 MBk6-compact-layer-031-0001.safetensors 74 MBk6-compact-layer-032-0001.safetensors 74 MBk6-compact-layer-033-0001.safetensors 74 MBk6-compact-layer-034-0001.safetensors 74 MBk6-compact-layer-035-0001.safetensors 74 MBk6-compact-layer-036-0001.safetensors 74 MBk6-compact-layer-037-0001.safetensors 74 MBk6-compact-layer-038-0001.safetensors 74 MBk6-compact-layer-039-0001.safetensors 74 MBk6-compact-layer-040-0001.safetensors 74 MBk6-compact-layer-041-0001.safetensors 74 MBk6-compact-layer-042-0001.safetensors 74 MBk6-compact-layer-043-0001.safetensors 74 MBk6-compact-layer-044-0001.safetensors 74 MBk6-compact-layer-045-0001.safetensors 74 MBk6-compact-layer-046-0001.safetensors 74 MBk6-compact-layer-047-0001.safetensors 74 MBk6-compact-layer-000-0001.safetensors 74 MBk6-compact-layer-001-0001.safetensors 74 MBk6-compact-layer-002-0001.safetensors 74 MBk6-compact-layer-003-0001.safetensors 74 MBk6-compact-layer-004-0001.safetensors 74 MBk6-compact-layer-005-0001.safetensors 74 MBk6-compact-layer-006-0001.safetensors 74 MBk6-compact-layer-007-0001.safetensors 74 MBk6-compact-layer-008-0001.safetensors 74 MBk6-compact-layer-009-0001.safetensors 74 MBthinkingmachines/Inkling-2-Small-MXFP8