Qwen 2.5 0.5B – Coding Expert (FFN-Pruned)
This model is a coding-specialized expert derived from Qwen 2.5 0.5B using activation-based FFN neuron pruning on the HumanEval dataset.
Key features
- ~25% FFN neuron pruning
- No retraining
- Stable Python code generation
- Reduced compute per token
Method
- Collected per-neuron FFN activations during inference
- Pruned lowest-activation neurons per layer
- Preserved embeddings, attention, and early representations
Intended use
- Research
- Model compression experiments
- Expert routing / MoE research
Limitations
- Specialized for coding
- General language performance may degrade
This is a research artifact, not a production model.