Granite-4.2-30B-Fable-Distill

ibm-granite/granite-4.2-30b fine-tuned on agentic coding traces and chat distilled from Claude Fable 5, with a multilingual slice. Merged bf16 weights (~55 GB), drop-in for the base model: same architecture, same tokenizer, same chat template. It is primarily a tool-use tune — three-quarters of the supervised tokens are tool calls — and it trains Granite's native <think> reasoning.

The LoRA adapter this was merged from is published separately as armand0e/Granite-4.2-30B-Fable-Distill-LoRA.

Serving with vLLM

Granite 4.2 uses <think>…</think> reasoning and an XML tool-call format, and vLLM (≤ 0.28) has no parser named for it. Two existing parsers match its wire format exactly:

• reasoning: deepseek_r1 — Granite's chat template prefills the opening <think> and the model only emits </think>; this parser is built for that shape (the granite parser targets Granite 3.x's prose markers and does not work).
• tools: qwen3_coder — Granite renders <tool_call><function=NAME><parameter=X>…</parameter></function></tool_call>, the same format as the Qwen3-Coder parser (the granite4 parser expects a different one).

vllm serve armand0e/Granite-4.2-30B-Fable-Distill \
  --served-model-name granite-fable \
  --reasoning-parser deepseek_r1 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --default-chat-template-kwargs '{"enable_thinking": true}' \
  --max-model-len 40960 --dtype bfloat16

On a single 64–80 GB GPU the bf16 weights leave little room for KV cache; add --quantization bitsandbytes for in-flight 4-bit, or quantize to AWQ/GPTQ INT4 for the best decode throughput.

Keep thinking enabled. With thinking off, Granite's template prefills an empty <think></think> and the model emits no tag at all, so deepseek_r1 — which treats everything before a </think> as reasoning — returns the whole reply in the reasoning field with empty content, and tool calls are not parsed. Send chat_template_kwargs: {"enable_thinking": true} (or set it as the server default as above). enable_thinking is the template's kwarg name; low_effort: true is also available for shorter reasoning.

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("armand0e/Granite-4.2-30B-Fable-Distill")
model = AutoModelForCausalLM.from_pretrained("armand0e/Granite-4.2-30B-Fable-Distill", torch_dtype="bfloat16", device_map="auto")
msgs = [{"role": "user", "content": "Explain what a race condition is in two sentences."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=512)[0, ids.shape[1]:]))

Training data

773 examples / 8.78M tokens after rendering at a 40,960-token context (trim_followups; median 1.5k tokens, p90 35.6k). Prepared with teich ≥ 0.3.6 (Granite 4.2 support), reasoning kept and supervised, per-row auto mode: 652 non-thinking / 121 thinking rows.

Supervised text by span: tool_call 73%, final_answer 20%, reasoning 8%. Language mix by tokens: English 90%, es/fr/de/pt/ja/zh ~10% combined.

Training

QLoRA (NF4 base), LoRA r=32 / α=32 / dropout 0 on q,k,v,o,gate,up,down_proj — 281M trainable parameters, merged back into the bf16 base weights. 40,960-token context, LR 8e-5 linear, warmup 5, 2 epochs (194 steps), batch 1 × grad-accum 8, paged_adamw_8bit, max_grad_norm 0.3. Single 64 GB GPU, ~24 h. Base-model loss on this data was 0.90 at step 1; per-step loss settled in the 0.69–0.75 band through epoch 1 and averaged 0.75 (median 0.73) across epoch 2.

Trained with Unsloth + TRL; masking and span audit by teich. Note for anyone reproducing on teich ≤ 0.3.7: Granite supervised spans could extend past their own assistant turn into the following user turn (24 real cases in this data); the training script clamps spans at a genuine turn boundary (<|im_end|> followed by <|im_start|>), while leaving quoted <|im_start|> strings inside code untouched.

Limitations

Not recommended for coding. Despite the agentic/tool-use training data, this model — like the base Granite 4.2 30B — is weak at writing and editing code. Use it for tool orchestration, extraction, chat and multilingual tasks; reach for a coding-focused model (e.g. a Qwen3 coder variant) when the task is primarily code.

The non-English share is ~10% of tokens; this is a lighter multilingual signal than the earlier Gemma-based Fable tunes. Supervising reasoning was chosen deliberately for a thinking-native model, but it has not been benchmarked against an answers-only variant of the same run. The adapter was trained against NF4-quantized base weights and merged into bf16; this is standard QLoRA practice but means the merged model is not bit-identical to what training saw.