Loadable ExLlamaV3 / REXL3 pack of nex-agi/Nex-N2.6
(Glm5NextForConditionalGeneration, glm5_next, 321B total / ~29B active MoE,
129 source shards in FP8) at 2.48 bits per weight, built to fit a single
128 GB-class unified-memory box at the full 262,144-token context with the
NextN speculative draft layer resident on device.
SAGE is our own mixed-precision quantization method for EXL3. The method is not published.
Footprint: 103,651,276,152 bytes (~96.5 GiB) device-resident, 15 shards, 103,721,369,128 bytes on disk. Highest Top-1 Agreement 95.52%, mean KLD 0.0246 against the FP8 source.
Greedy top-1 token agreement and mean forward KL divergence against the FP8 source checkpoint, float64 logits, full vocabulary, assistant-turn token mask.
| Metric | Value |
|---|---|
| Top-1 Agreement | 95.52% |
| Mean KLD | 0.0246 |
| Base model | nex-agi/Nex-N2.6 @ 987472298e7867721576ac49a253a6f5b526013e |
| Quant method | EXL3 (quant_method: "exl3") |
| Bits per weight | 2.48 |
| Device-resident | 103,651,276,152 B (96.53 GiB) |
| Shards | 15 (model-00001 … model-00015) |
| Architecture | Glm5NextForConditionalGeneration (48-layer hybrid KDA/MLA MoE + NextN draft) |
| Context | 262,144 tokens |
Tensors use different EXL3 K values; the kernels read K from each trellis
tensor at load. If a tensor is EXL3, its K is self-describing on the trellis.
from exllamav3 import Model, Config, Cache, Tokenizer
model = Model.from_directory("Nex-N2.6-SAGE-REXL3-2.48bpw")
config = Config.from_model(model)
cache = Cache(config)
tokenizer = Tokenizer.from_model(model)
SAGE is our own mixed-precision quantization method for EXL3. The method is not published.
This pack is a quantization of nex-agi/Nex-N2.6. All model weights, tokenizer
files, chat template and architecture definitions originate from that release;
only the numerical precision of the weight tensors was changed.
Distributed under the same license as the source model. See the source model card for the full terms.
model-00005-of-00015.safetensors 9.6 GBmodel-00013-of-00015.safetensors 9.0 GBmodel-00009-of-00015.safetensors 9.0 GBmodel-00002-of-00015.safetensors 8.0 GBmodel-00004-of-00015.safetensors 7.4 GBmodel-00014-of-00015.safetensors 6.8 GBmodel-00008-of-00015.safetensors 6.8 GBmodel-00007-of-00015.safetensors 6.8 GBmodel-00010-of-00015.safetensors 6.8 GBmodel-00012-of-00015.safetensors 6.8 GBmodel-00003-of-00015.safetensors 6.8 GBmodel-00011-of-00015.safetensors 6.7 GBmodel-00006-of-00015.safetensors 6.7 GBmodel-00015-of-00015.safetensors 4.0 GBmodel-00001-of-00015.safetensors 2.6 GB