SLIMDER Qwen3.8 REAP-384 depth-36 (macroblocks 2, 4, 5 removed)
This is the promoted BF16 depth-36 structural checkpoint from the S3
progressive-reduction screen. It derives from the public depth-40 REAP-384
checkpoint at revision 2835be64b0429eddef35c001b818929223476c5e.
The parent had already removed original macroblocks 4 and 5. S3 removed local depth-40 macroblock 2, which is original macroblock 2 (source layers 8-11). The resulting model therefore removes original macroblocks 2, 4, and 5.
Validation evidence is public in
sjakek/slimder-qwen38-s3-results-20260831:
- ten-candidate runtime screen: residual cosine 0.943, top-1 agreement 0.805, loss increase 0.138;
- disjoint holdout: residual cosine 0.945, top-1 agreement 0.798, loss increase 0.052, with no catastrophic category outlier;
- repaired executable functional gate: 8/8 tasks, versus 7/8 for the unchanged depth-40 parent, including executed Python, exact JSON tool calls, retrieval, and general reasoning.
The checkpoint contains 36 transformer layers and 123,174,648,480 parameters. The routed-expert width remains REAP-384. A Q4_K_M deployment artifact is planned from this exact promoted checkpoint.
Qwen3.8 Perian project lineage
This repository is retained in the Qwen3.8 Perian checkpoints collection. Its exact position in the lineage is: Depth-pruning precursor at 36 layers. It retains 384 experts per layer and the full PLE table and predates the Perian QLoRA.
The final Qwen3.8 Perian GGUF release combines three reductions and one post-training stage:
- depth: 48 to 32 transformer layers;
- routed-expert width: 384 to 288 experts per layer;
- PLE n-gram capacity: 320,001,446 to 160,000,768 rows (50%, about 25.60B parameters removed), using activation-aware bigram and frequency-ranked trigram selections validated on a document-disjoint 5M-token holdout;
- rank-32 QLoRA on 12,558 normalized traces spanning math/STEM reasoning, coding/debugging, agentic tool use, retrieval, and general multi-step reasoning. The trace mixture draws from several frontier-model families, including Fable 5, GLM 5.2, Kimi K3, Claude Opus 4.7, Qwen3.8-Max, and GPT-5.6-Sol. The final merged milestone was trained through 9,336,692 supervised assistant tokens.
Earlier checkpoints in this collection do not inherit later stages merely by being listed beside them; the stage statement above is authoritative for this artifact.