Whittle 14.7B logit-distillation adapter (research preview)

LoRA r32 trained to match the output distribution of the Qwen3.8-27B parent, not just the reference answers. Targets are the parent's top-128 log-probs per token (recorded once over the heal corpus), and the loss is KL against those plus a small cross-entropy anchor, taken on assistant turns only.

Why distribution rather than labels: the base model already predicts this corpus almost perfectly (cross-entropy ~4e-05) while its distribution sits ~2.5 nats from the parent's. Hard labels had nothing left to teach; the distribution did.

3.90M tokens, ~1.2 epochs, 80 minutes on one A100. The teacher pass ran the parent at nf4, so the deep tail of the targets is approximate.

Research preview, needs further work. Published as a record of method and measurements. See the Whittle collection for the models themselves.