*** WORK IN PROGRESS ***
Qwen3.8-27B-QAD-E1
A mixed NVFP4/MXFP8 quantization-aware distillation of Qwen3.8-27B, trained for one epoch.
The student learns from the original BF16 teacher while its MLP weights are quantized in the forward pass. Distillation updates the MLP weights and text normalization weights to account for quantization error. This is a trained distillation checkpoint, not a post-training conversion of the original weights.
Precision
| Component | Representation |
|---|---|
| MLP gate, up and down projections | NVFP4, 16-element blocks |
| Attention and gated-delta-network projections | MXFP8, 32-element blocks |
| LM head | MXFP8, 32-element blocks |
| Trained text normalization weights | FP32 masters |
| Token embeddings, GDN convolutions and dynamics | Original BF16 |
| Vision encoder and remaining source tensors | Unchanged |
Attention/GDN projections and the LM head were frozen in their MXFP8 representations during distillation. Packed NVFP4 and MXFP8 weights reconstruct to the same BF16 weight values used by the student during training. The tokenizer, chat template, generation configuration and multimodal processors are retained from the base model.
Activation calibration
MLP activation scales use the p99.999 token-row maximum from 390,497,191 raw-text and chat tokens. Each token contributes its maximum absolute input value across channels; an exact BF16 histogram records these row maxima. Raw and chat histograms are pooled before selecting the quantile.
Within each of the 64 dense layers, gate (w1) and up (w3) have separate
scale tensors containing the same value. Down (w2) has an independent
scale. These are dense projections; there is no expert axis.
Calibration uses the trained weight representations with BF16 activations. The deployment configuration specifies calibrated NVFP4 MLP activations and dynamic MXFP8 attention/head activations. Serving kernels therefore introduce activation quantization beyond the training forward pass.
Format
Hugging Face safetensors with ModelOpt mixed-precision quantization metadata. The runtime must support the base model architecture, NVFP4 dense linears and MXFP8 linears. Packed-weight reconstruction and export integrity are checked; downstream evaluation and serving-quality qualification are separate.
License
Apache 2.0, following the base model.