Granite 4.2 3B + Uno drafter adapter (GGUF)
Standard GGUF files for ibm-granite/granite-4.2-3b and its Uno drafter adapter. Training details and TPF results are in Cyronius/granite-4.2-3b-uno. The NPU build is Cyronius/Granite-4.2-3B-Uno-NPU2.
Files
| File | Contents |
|---|---|
granite-4.2-3b-Q4_1.gguf |
Base model, Q4_1 (5.11 BPW). Same quantization as the NPU build. |
granite-4.2-3b-Q4_K_M.gguf |
Base model, Q4_K_M (4.90 BPW). |
granite-4.2-3b-uno-f32.gguf |
Uno adapter as a GGUF LoRA: rank 128, alpha 8192, q/k/v/o/gate/up/down of all 40 layers, F32. |
The base files are made from IBM's bf16 GGUF with llama-quantize (llama.cpp bb4caa75). The adapter is made with
convert_lora_to_gguf.py.
Limits in llama.cpp
The base files run in llama.cpp as ordinary Granite 4.2 3B.
The adapter does not give Uno decoding in llama.cpp. Uno needs the adapter on the noise rows only, with the
base model verifying the draft with the adapter off. llama.cpp applies a LoRA to every token and has no field for the
noise ids (uniform in [1, 100256)). The adapter was trained for 8-row blocks. A runtime that routes the adapter per
row is needed.
Credits
- Method and training code: Uno by IFM (Apache-2.0).
- Base model: IBM Granite 4.2 3B (Apache-2.0).
- Compute: AMD Developer Cloud.