Granite-4.2-30B-Fable-Distill-LoRA

The QLoRA adapter (PEFT, r=32) from a fine-tune of ibm-granite/granite-4.2-30b on agentic coding traces and chat distilled from Claude Fable 5, with a multilingual slice. It is primarily a tool-use tune — three-quarters of the supervised tokens are tool calls — and it trains Granite's native <think> reasoning.

Serving with vLLM

Granite 4.2 uses <think>…</think> reasoning and an XML tool-call format, and vLLM (≤ 0.28) has no parser named for it. Two existing parsers match its wire format exactly:

• reasoning: deepseek_r1 — Granite's chat template prefills the opening <think> and the model only emits </think>; this parser is built for that shape (the granite parser targets Granite 3.x's prose markers and does not work).
• tools: qwen3_coder — Granite renders <tool_call><function=NAME><parameter=X>…</parameter></function></tool_call>, the same format as the Qwen3-Coder parser (the granite4 parser expects a different one).

vllm serve ibm-granite/granite-4.2-30b \
  --quantization bitsandbytes \
  --enable-lora --lora-modules granite-fable=armand0e/Granite-4.2-30B-Fable-Distill-LoRA --max-lora-rank 32 \
  --served-model-name granite-fable \
  --reasoning-parser deepseek_r1 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --default-chat-template-kwargs '{"enable_thinking": true}' \
  --max-model-len 40960 --dtype bfloat16

A merged bf16 checkpoint is published separately as armand0e/Granite-4.2-30B-Fable-Distill; use that if you want to serve without LoRA or to quantize.

Keep thinking enabled. With thinking off, Granite's template prefills an empty <think></think> and the model emits no tag at all, so deepseek_r1 — which treats everything before a </think> as reasoning — returns the whole reply in the reasoning field with empty content, and tool calls are not parsed. Send chat_template_kwargs: {"enable_thinking": true} (or set it as the server default as above). enable_thinking is the template's kwarg name; low_effort: true is also available for shorter reasoning.

The adapter was trained against an NF4 (bitsandbytes) base, so serving it unmerged on --quantization bitsandbytes reproduces the training-time weights; a 4-bit base also leaves far more KV cache than the 55 GB bf16 checkpoint.

Training data

773 examples / 8.78M tokens after rendering at a 40,960-token context (trim_followups; median 1.5k tokens, p90 35.6k). Prepared with teich ≥ 0.3.6 (Granite 4.2 support), reasoning kept and supervised, per-row auto mode: 652 non-thinking / 121 thinking rows.

Supervised text by span: tool_call 73%, final_answer 20%, reasoning 8%. Language mix by tokens: English 90%, es/fr/de/pt/ja/zh ~10% combined.

Training

QLoRA (NF4 base), LoRA r=32 / α=32 / dropout 0 on q,k,v,o,gate,up,down_proj — 281M trainable parameters. 40,960-token context, LR 8e-5 linear, warmup 5, 2 epochs (194 steps), batch 1 × grad-accum 8, paged_adamw_8bit, max_grad_norm 0.3. Single 64 GB GPU, ~24 h. Base-model loss on this data was 0.90 at step 1; per-step loss settled in the 0.69–0.75 band through epoch 1 and averaged 0.75 (median 0.73) across epoch 2.

Trained with Unsloth + TRL; masking and span audit by teich. Note for anyone reproducing on teich ≤ 0.3.7: Granite supervised spans could extend past their own assistant turn into the following user turn (24 real cases in this data); the training script clamps spans at a genuine turn boundary (<|im_end|> followed by <|im_start|>), while leaving quoted <|im_start|> strings inside code untouched.

Limitations

Not recommended for coding. Despite the agentic/tool-use training data, this model — like the base Granite 4.2 30B — is weak at writing and editing code. Use it for tool orchestration, extraction, chat and multilingual tasks; reach for a coding-focused model (e.g. a Qwen3 coder variant) when the task is primarily code.

The non-English share is ~10% of tokens; this is a lighter multilingual signal than the earlier Gemma-based Fable tunes. Supervising reasoning was chosen deliberately for a thinking-native model, but it has not been benchmarked against an answers-only variant of the same run.