bonsai: depth0/experts8
Pruned and distilled from Qwen3-30B-A3B-Base for TypeScript code completion. Part of the bonsai project, which applies the Minitron structured-pruning and knowledge-distillation recipe (Muralidharan et al. 2024, Sreenivas et al. 2024) to a Mixture-of-Experts model, a pruning axis neither paper covers.
What was pruned
- Depth: 0 of 48 layers removed (48 kept)
- Experts: 8 of 128 routed experts removed per layer (120 kept)
- Result: ~28.72B parameters (94.2% of baseline's ~30.5B)
Training
500-step fixed-loss short schedule (75-step warmup, peak LR 1e-4), distilled against the uncorrected teacher. Best-validation checkpoint, step 29.
Results
HumanEval-TS (TypeScript MultiPL-E), full 159-problem suite, temperature 0.2, top-p 0.95:
| pass@1 | |
|---|---|
| Qwen3-30B-A3B-Base (unpruned) | 0.717 (114/159) |
| This model | 0.704 (112/159) |
98.2% of baseline accuracy at 94.2% of baseline's parameters.
Real serving throughput, measured with vLLM and SGLang, 159 HumanEval-TS prompts, 512 fixed output tokens/request, concurrency 16, 1x B300:
| vLLM (tok/s) | SGLang (tok/s) | |
|---|---|---|
| Qwen3-30B-A3B-Base (unpruned) | 2545.6 | 2690.0 |
| This model | 2535.4 | 2730.5 |
Notes
Closest to the unpruned baseline of any candidate in this project (98.2% accuracy retention), but only 5.8% smaller. Use this one when accuracy parity matters more than size; otherwise see depth0/experts64, the project's recommended default.
Project
Other checkpoints from this sweep: bonsai collection. Code and full writeup: github.com/SartajBhuvaji/bonsai (private repo).