REAM-288 Mixed NVFP4/FP8 Recovery LoRA v2

This repository contains the promoted recovery adapter for jakeatx/slimder-qwen38-ream288-depth32-agentic-ngram50-mixed-nvfp4-v2.

The root adapter is the 50k-token checkpoint blended at 62.5% of its trained LoRA strength (r=8, trained alpha=16, promoted alpha=10). This was the strongest tested blend that passed every sealed behavior-preservation gate.

Promoted adapter results

• Recovery tokens: 50,176 non-benchmark tokens
• Trainable parameters: 10,158,080
• Baseline sealed loss: 4.258437
• Promoted sealed loss: 3.958144 (improvement 0.300293)
• Token KL: 0.176316 (maximum 0.20)
• Router top-10 agreement: 0.924538 (minimum 0.90)
• PLE cosine: 1.000004 (minimum 0.98)
• Qualitative catastrophic failures: 0
• Gate result: passed

The 250k training milestone also improved held-out validation loss from 1.773725 at 50k to 1.596309, but it was not promoted: its sealed token KL was 1.002711, router agreement was 0.834931, and it produced one Japanese character-loop failure. The full-strength 50k adapter was also not promoted because token KL was 0.329827. These checkpoints and their evidence are kept under research-checkpoints/ for reproducibility, not as recommended defaults.

Training

• Sequence length: 512
• Learning rate: 2e-5
• LoRA rank: 8
• Trained LoRA alpha: 16
• Promoted LoRA alpha: 10
• Training corpus: 500 recovery rows plus 10 held-out validation rows
• AAII, Frontier, AgentX, and other evaluation questions were excluded

Usage

Load the mixed v2 base checkpoint with its included custom runtime, then apply the adapter in this repository with PEFT. The base uses experimental qwen4_exp, compact PLE remapping, NVFP4 routed experts, and rowwise FP8 PLE storage, so it is not a generic drop-in Transformers checkpoint.

Evidence

evidence/ contains the sealed comparison reports, training state, runtime validation, precision audits, and hashes. The strengthened qualitative checker detects both whitespace-token repetition and repeated non-ASCII character loops.

Limitations

This is an experimental recovery adapter. It passed the bounded sealed gate, but broad task and safety evaluation are still required before production use.

Qwen3.8 Perian project lineage

This repository is retained in the Qwen3.8 Perian checkpoints collection. Its exact position in the lineage is: Early bounded recovery-QLoRA pilot on the fully compressed mixed base; it predates the final post-training run.

The final Qwen3.8 Perian GGUF release combines three reductions and one post-training stage:

• depth: 48 to 32 transformer layers;
• routed-expert width: 384 to 288 experts per layer;
• PLE n-gram capacity: 320,001,446 to 160,000,768 rows (50%, about 25.60B parameters removed), using activation-aware bigram and frequency-ranked trigram selections validated on a document-disjoint 5M-token holdout;
• rank-32 QLoRA on 12,558 normalized traces spanning math/STEM reasoning, coding/debugging, agentic tool use, retrieval, and general multi-step reasoning. The trace mixture draws from several frontier-model families, including Fable 5, GLM 5.2, Kimi K3, Claude Opus 4.7, Qwen3.8-Max, and GPT-5.6-Sol. The final merged milestone was trained through 9,336,692 supervised assistant tokens.

Earlier checkpoints in this collection do not inherit later stages merely by being listed beside them; the stage statement above is authoritative for this artifact.