Mistral-Large-4-REAP-25-UD-IQ1_S-GGUF (Experimental / unvalidated)

[!WARNING] Experimental prototype — model quality and speed have not been established.

This is a trial release shared for investigation and independent testing. It has not undergone meaningful task-quality evaluation. Coding, reasoning, mathematics, tool use, normal conversation, long-context quality and generation speed are unverified. Substantial regressions are possible.

The only real-model checks completed are a tiny synthetic-text likelihood diagnostic (8 passages, 435 next-token likelihood positions) and loading plus one generated token. The slightly lower PPL on that tiny set is not evidence of preserved or improved real-world accuracy. Reduced file size is not evidence of faster generation. Upstream benchmark scores do not apply to this derivative.

お試しの実験版です。十分な精度評価は行っておらず、実用上の精度維持・向上や高速化は確認できていません。 小規模な尤度検査と1tokenの生成確認のみで、会話・コード・推論などの品質は未検証です。 大きな品質低下もあり得る前提で、独立した検証用に公開しています。

An experimental, quantized and structurally pruned derivative of Mistral Large 4. Mistral created the original model; Unsloth created the source UD-IQ1_S quantization. This repository contributes layerwise expert selection and a smaller GGUF export. It is not a model trained from scratch, a new quantization, or an official Mistral/Unsloth release.

Changes

• Each of the 71 MoE layers retains 96 of its original 128 routed experts (25% removed).
• The router still executes four routed experts per token. The shared expert is preserved.
• All 72 decoder layers, attention weights, dense layers, embeddings, output head, tokenizer and chat template remain.
• Retained quantized expert blocks are copied byte for byte. Router rows are selected in the same order.
• No finetuning, expert merging or weight requantization is used. Calibration uses quantized CPU inference kernels.
• The text-model tensors contain approximately 792 billion parameters after pruning.
• Expected model-file size is about 167.13 GiB, versus 217.01 GiB for the source.

UD-IQ1_S describes the inherited mixed quantization recipe; it does not mean every tensor uses one bit. The upstream A52B active-parameter designation is retained as an approximate architecture designation: reducing the expert pool does not reduce the number of experts executed per token.

Method and calibration

Router-weighted Expert Activation Pruning (REAP) scores expert j by the mean of router_weight * L2(expert_output) over tokens that select j, not over all tokens. This implementation measures actual activations of the existing quantized CPU model. It is an adaptation of REAP, not the official Cerebras or Unsloth pruning implementation.

Calibration proceeds layer by layer. After scoring a layer across the entire calibration set, it removes the 32 lowest-scoring observed experts, reroutes among the 96 retained experts, renormalizes the top-four weights and recomputes that layer's FFN output before proceeding. Unobserved experts are protected rather than assigned a zero importance score.

Calibration uses 32 original, manually authored synthetic text passages (2,744 tokens total), truncated at 128 tokens per passage, with no chat-template wrapping. Topics include Japanese, programming, mathematics, tool/JSON conventions, explanations and several additional languages. The small calibration set is not representative of all capabilities or long contexts. The passages, exact selected expert IDs, counts and score sums are included for inspection.

Source quantization repository: unsloth/Mistral-Large-4-1T-A52B-GGUF. Source revision: 1eb4373d37e76f6e7b4b8efa20e1b1ca40b3b964. Source files: UD-IQ1_S/Mistral-Large-4-1T-A52B-UD-IQ1_S-00001-of-00006.gguf through shard 6. Exact source/output SHA-256 hashes and the old-to-new expert mapping are in manifest.json and reap.json.

Evaluation status

Experimental candidate. A held-out diagnostic on eight original text passages (443 input tokens, 435 next-token likelihood positions, maximum 64 tokens per passage) gave:

The aggregate perplexity ratio on these eight passages is 0.992409. This is a tiny diagnostic only; it does not support a claim that accuracy was preserved or improved. The models' top-one predictions agreed at 88.71% of the 443 input positions. That is agreement between models, not a task-accuracy score, and their generated answers need not match. The evaluation uses the same quantized CPU kernels and F32 KV cache for both models. Calibration and held-out text are separate. See evaluation/evaluation.json and raw records for exact results. These numbers do not establish a statistically significant improvement or general quality preservation.

An ML4-capable llama.cpp build (52e53a76df0298d2c316ae7e704eeec03b0bf478) loaded all six shards and generated one token successfully on CPU with F32 KV cache. The smoke test used raw token IDs [1,1035], not a conversational question. Large mmap prefetch was suppressed by an existing I/O-only demand-paging helper to protect the RAM-limited test machine. See validation/llama-validation.json for details.

The normal greedy chat/latency comparison stopped on memory pressure while processing the original model's first prompt, before producing a completed record. There is no measured normal-generation speedup or completed conversational quality comparison for this release. Unsloth Studio GUI loading, GPU inference, long contexts and vision were not validated. Loading successfully, matching calibration simulation, or improving perplexity on a small set does not establish preserved coding, reasoning, multilingual, long-context or tool-use capability. Do not interpret reduced disk size as an equal percentage reduction in latency.

No vision projector is included or evaluated. This artifact is evaluated for text only. Published upstream model benchmark scores must not be attributed to this pruned derivative.

Loading in Unsloth Studio

Use an ML4-capable llama.cpp engine in Unsloth Studio. An older engine that does not recognize mistral-large4 cannot load this model. Keep all six GGUF shards in one directory and select the 00001-of-00006.gguf file through Studio's local-model folder picker. The tokenizer and original chat template are embedded; no custom runtime pruning patch is required. The installed Studio engine on the test machine (build 11539, commit 1cdd47c40) rejected the original ML4 architecture too, so updating to an ML4-capable engine is required there.

This remains a very large model. 167 GiB of model files do not fit entirely in 123 GiB RAM plus a 24 GiB GPU; CPU/SSD offloading and additional cache/runtime memory are still relevant. Do not enable settings that eagerly copy all model weights into RAM on an undersized machine.

Attribution and license

Original model: Mistral AI. Original quantization: Unsloth. Pruning method: REAP paper.

The original Mistral Ecosystem Open-Weight License and attribution notice are provided in LICENSE. This derivative retains that license; publishing it under another account does not remove upstream attribution or the license conditions. See the license itself for its terms.