Qwen3.5-4B-M8

3-model DARE-TIES variant of Qwen3.5-4B, distilling Claude Opus 4.6 reasoning style from two community fine-tunes onto the official base.

The M8 recipe was the winner of a 13-variant ablation study (M1–M8 + sub-variants) measuring HumanEval and MBPP across different importance signals (Fisher, LRP, hybrid), gating mechanisms (PR #682 turbo head, mergetime detectors), and merge methods (DARE-TIES, OmniMerge v2, mergekit ex-LRP).

M8 is the simplest production-recommended config: it matches the best balanced merges on HE+MBPP without requiring expensive Fisher / LRP precomputation. It uses only magnitude-based importance combined with all gating mechanisms.

M8 Recipe

method:        omnimerge_v2
features:      obim, darex, emr
density:       0.53
weights:       0.55 / 0.45 (jackrong / crow)
darex-q:       0.75
importance:    magnitude (no Fisher, no LRP)
--pr682-turbo: critical-layer protect (norm/embed/head/bias forced density=1.0)
               + skip on shape mismatch
--m7-detector --m7-layer-aware:
               mergetime detector with layer-aware norm clamping
               (τ_norm=0.02 for attn, 0.10 for MLP)
               + sign-flip gate + consensus bonus
seed:          42

Run via:

python scripts/dare_ties_merge.py \
    --base Qwen/Qwen3.5-4B \
    --source Jackrong/Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2 \
    --source Crownelius/Crow-4B-Opus-4.6-Distill-Heretic_Qwen3.5 \
    --output ./m8 \
    --method omnimerge_v2 --v2-features obim,darex,emr \
    --weights 0.55,0.45 --density 0.53 --darex-q 0.75 \
    --pr682-turbo \
    --m7-detector --m7-layer-aware \
    --skip-patterns "model.visual,mtp.layers" \
    --seed 42 --device cuda

Evaluation

All evaluations: lm-evaluation-harness, llama.cpp llama-server with Q6_K quantization, --reasoning-format deepseek --reasoning-budget 8192, /v1/completions raw endpoint, greedy decoding (temperature=0.0, top_p=1.0), max_gen_toks=2048, --parallel 2, fence-strip rescore.

Headline numbers (Q6_K)

M8 retains ≈97% of source 1's HE and beats both sources on MBPP.

Full ablation matrix

13 variants tested. All use same base + 2 sources, same eval pipeline.

Why M8

1. Importance signals (Fisher / LRP / hybrid) contribute ≈zero on this base/source pair at density 0.53. M4-v2 and M6-hybrid produced byte-identical merged tensors at d=0.7 — top-70% importance masks coincide regardless of signal source.
2. Layer-aware mergetime gating is what moves MBPP from 49 → 51. M7v2 shows it; M8 confirms the gain holds without any importance signal.
3. Critical-layer protection (--pr682-turbo) keeps norm / embed / head / bias at density 1.0, preventing per-token / per-vocab structure from being shredded by sparsification.
4. M7's HE 59.76 / MBPP 44.20 is not the production pick: the uniform τ=0.02 clamp damages MBPP. Layer-aware clamping (τ_attn=0.02, τ_mlp=0.10, M7v2 + M8) recovers MBPP at small HE cost.

So M8 is the stable recipe: matches M4v2-dare on MBPP (51.40 vs 51.80) and is within 0.6 pp on HE (54.27 vs 54.88), while requiring no external signal precomputation.

Files

• model-*.safetensors — BF16 merged weights
• config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja, merges.txt, vocab.json — Qwen3.5 tokenizer and config (unchanged from base)

Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "ManniX-ITA/Qwen3.5-4B-M8",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained("ManniX-ITA/Qwen3.5-4B-M8")

msgs = [{"role": "user", "content": "Write a Python function that returns the n-th Fibonacci number."}]
inputs = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True).to(model.device)
out = model.generate(inputs, max_new_tokens=1024, do_sample=False)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

llama.cpp (Q6_K)

llama-server -m Qwen3.5-4B-M8-Q6_K.gguf \
    --port 8099 -c 32768 -ngl 99 --no-warmup \
    --reasoning-format deepseek --reasoning-budget 8192 \
    --parallel 2 --cache-type-k q8_0 --cache-type-v q8_0

Upstream contributions

Two patches submitted to mergekit PR #682 during this study:

• Comment #4361070820 — mergekit/merge_methods/lrp.py silently fell back to magnitude when given .safetensors LRP score files (torch.load raises on safetensors). Added a safetensors loader branch.
• Comment #4361416737 — mergekit/architecture/auto.py _wi() checked only layer 0 for optionality, breaking hybrid architectures (Qwen3.5 alternating self_attn / linear_attn). Fixed to check all layer instantiations.

These two patches together unblock mergekit-yaml end-to-end on Qwen3.5 hybrid models with the LRP merge method.

License

Inherits the Apache 2.0 license from Qwen/Qwen3.5-4B. Source models retain their own licenses.

Acknowledgements

• Alibaba / Qwen team for Qwen3.5-4B
• Jackrong and Crownelius for the Claude Opus 4.6 distillation source models
• @Tusm11 and Arcee AI for mergekit PR #682 (ex-LRP merge method)