The routed experts of DeepSeek-V4.1-Flash quantized with GRAFT (Grid-Aligned FP4 Trellis) at 3.006 bits per weight. GRAFT codes MXFP4 checkpoints below 4 bits by keeping the source grid instead of rotating it away, and it decodes back to MXFP4 blocks with a per-column scale and a sparse tail correction.
Paper: GRAFT: Grid-Aligned FP4 Trellis Quantization for FP4-Native Language Models
| GRAFT (this repo) | EXL3 3 bpw | |
|---|---|---|
| held-out expert error (relative to EXL3) | 0.63x | 1 |
| excess NLL over the native model (nats) | 0.0085 | 0.0131 |
| KL to the native model | 0.0369 | 0.0469 |
| top-1 agreement | 92.5% | 91.7% |
model-00001 ... model-00050-of-00051.safetensors: every layer's routed experts in GRAFT-MX format, plus the
layers' other weights. model-00051-of-00051.safetensors and every tensor outside the routed experts are
byte-identical to coolbho3k/DeepSeek-V4.1-Flash-EXL3-3bpw.config.json: quantization_config.codebook = "graft-mx"; quantization_config.graft.flagged lists the matrices
that carry block flags.draft/ and engrams/ hold only the configs and indexes. The drafter weights and the engram tables are unchanged
from the EXL3 repository; copy them from there into the same folders.Each routed-expert matrix layers.L.ffn.experts.E.wN (N = 1, 2, 3) is stored as:
| tensor | dtype | shape | content |
|---|---|---|---|
trellis |
int16 | [k/16, n/16, 48] |
one 768-bit trellis bitstream per 16x16 tile (read as little-endian 32-bit words) |
suh |
float16 | [k] |
header: the first 16 values reinterpreted as 8 float32 (qa0, qb0, d, 0, qa1, qb1, 0, 0), repeated |
svh |
float16 | [n] |
per-output-column scale |
mul1 |
int32 | scalar | hash multiplier 0x83DCD12D |
graft_flags |
int32 | [k/32, n/32] |
flagged matrices only: bit c mod 32 of word [r/32, c/32] flags the 32-row block of column c |
k is the input dimension and n the output dimension, so the decoded matrix is the transpose of the original
weight tensor. A reference decoder will be published separately.
MIT, as the base model (Copyright (c) 2023 DeepSeek; see LICENSE).
model-00002-of-00051.safetensors 4.3 GBmodel-00043-of-00051.safetensors 4.3 GBmodel-00042-of-00051.safetensors 4.3 GBmodel-00003-of-00051.safetensors 4.3 GBmodel-00041-of-00051.safetensors 4.3 GBmodel-00038-of-00051.safetensors 4.3 GBmodel-00030-of-00051.safetensors 4.3 GBmodel-00039-of-00051.safetensors 4.3 GBmodel-00044-of-00051.safetensors 4.3 GBmodel-00046-of-00051.safetensors 4.3 GBmodel-00001-of-00051.safetensors 4.3 GBmodel-00015-of-00051.safetensors 4.3 GBmodel-00037-of-00051.safetensors 4.3 GBmodel-00011-of-00051.safetensors 4.3 GBmodel-00045-of-00051.safetensors 4.3 GBmodel-00022-of-00051.safetensors 4.3 GBmodel-00004-of-00051.safetensors 4.3 GBmodel-00050-of-00051.safetensors 4.3 GBmodel-00040-of-00051.safetensors 4.3 GBmodel-00016-of-00051.safetensors 4.3 GBmodel-00014-of-00051.safetensors 4.3 GBmodel-00033-of-00051.safetensors 4.3 GBmodel-00010-of-00051.safetensors 4.3 GBmodel-00009-of-00051.safetensors 4.3 GBmodel-00047-of-00051.safetensors 4.3 GBmodel-00023-of-00051.safetensors 4.3 GBmodel-00027-of-00051.safetensors 4.3 GBmodel-00031-of-00051.safetensors 4.3 GBmodel-00049-of-00051.safetensors 4.3 GBmodel-00018-of-00051.safetensors 4.3 GBmodel-00017-of-00051.safetensors 4.3 GBmodel-00020-of-00051.safetensors 4.3 GBmodel-00005-of-00051.safetensors 4.3 GBmodel-00007-of-00051.safetensors 4.3 GBmodel-00021-of-00051.safetensors 4.3 GBmodel-00012-of-00051.safetensors 4.3 GBmodel-00026-of-00051.safetensors 4.3 GBmodel-00025-of-00051.safetensors 4.3 GBmodel-00036-of-00051.safetensors 4.3 GBmodel-00006-of-00051.safetensors 4.3 GBmodel-00008-of-00051.safetensors 4.3 GBmodel-00032-of-00051.safetensors 4.3 GBmodel-00034-of-00051.safetensors 4.3 GBmodel-00028-of-00051.safetensors 4.3 GBmodel-00048-of-00051.safetensors 4.3 GBmodel-00019-of-00051.safetensors 4.3 GBmodel-00013-of-00051.safetensors 4.3 GBmodel-00035-of-00051.safetensors 4.3 GBmodel-00029-of-00051.safetensors 4.3 GBmodel-00024-of-00051.safetensors 4.3 GBmodel-00051-of-00051.safetensors 687 MB有文件?提交种子。
提交种子