Groot-19B-A3B — TTQ quant set

TTQ (TokenTinkerer Quant) is an eleven-rung GGUF ladder for GrootTeam/Groot-19B-A3B-261023, from 3.56 bpw to 8.50 bpw (7.88 → 18.59 GiB), plus a Q8_0 vision projector for the model's 27-block ViT. Every rung uses our TTQ recipe — the same per-tensor shape we built for the Qwen3.6-35B-A3B TTQ ladder and then replayed onto the Cyber-Tiel TTQ ladder, now applied to Groot by exact tensor name — because Groot and Cyber-Tiel share the identical 753-tensor name set. Every rung was verified against the expected per-tensor map from our Cyber-Tiel build: 10/10 exact. Every number below is a measurement on this box, not an estimate.

The source is better than any TTQ set has had before. These rungs are quantized directly from a BF16 conversion of the original model (GrootTeam/Groot-19B-A3B-261023, 38.9 GB, 753 tensors, sha256 4c69e32f…), which we built and verified tensor-by-tensor against the safetensors payload — bit-identical body, exact F32 upcasts, documented V-head reorder. There is no requantization step anywhere in this ladder, so unlike our Cyber-Tiel set every rung inherits no upstream quantization noise. The BF16 file is also the KLD reference for every row below, and it ships in this repo as Groot-19B-A3B-261023-BF16.gguf (35.4 GiB).

---

The ladder

KLD is KL divergence against the BF16 reference's logits, measured over 40 fixed chunks of 512 tokens (lower is better; ± is the standard error over chunks). top-1 is same-top-token agreement. Two corpora, fixed and identical for every row: code (70% code / 10% diff / 10% terminal / 10% config from a frozen agentic eval split) and wikitext-2 test. Rows in bold are this set; the last row is upstream's only published GGUF, measured by the same protocol on the same box so the columns are directly comparable.

Full filenames are Groot-19B-A3B-TTQ-<bpw>bpw-<type>.gguf. Total 124.75 GiB for the eleven rungs, plus a 586 MiB projector.

Read it against upstream. The author publishes exactly one GGUF, Q4_K_M at 10.86 GiB. This ladder brackets it from both sides:

• Groot-19B-A3B-TTQ-4.34bpw-IQ4_XS.gguf (9.60 GiB) is 11.6% smaller and measures the same code KLD (0.02592 vs 0.02578 — a 0.1 σ difference) with better prose (0.0289 vs 0.0331).
• Groot-19B-A3B-TTQ-4.70bpw-IQ4_NL.gguf (10.39 GiB) is 4.3% smaller with 38% lower code KLD and 35% lower wikitext KLD.
• Everything from 5.05 bpw upward is strictly more accurate, and bigger.

How to read the names

Two independent fields:

• <bpw>bpw is what llama-quantize reported for that rung — the number the filename carries, the same convention as our other TTQ ladders. It runs ~2.5% above the rate you get by dividing the file's bytes by 19.43 B (3.48 for the 3.56bpw file, 8.22 for the 8.50bpw one): the quantizer measures the tensors it quantizes, while the file also carries the 301 F32 tensors. The GiB column is exact — it is stat on the file.
• The trailing token is the file's declared general.file_type and is distinct across the whole set — if two files declare the same ftype the Hub's download selector collapses them.

---

Which rung should I use?

---

Does the top of the ladder pay?

No, and this is the useful result. Q8_0 costs +24% size over Q6_K (14.96 → 18.59 GiB) and returns −11% code KLD (0.00508 → 0.00453) and −13% wikitext (0.00864 → 0.00752). The two code means differ by ~2 σ: real, but small.

Q8_0 was built deliberately as the ladder's maximum-quality cap, and it earns its place as a practically-lossless artifact to compare against — not as a quality upgrade. Q6_K is where this model's quantization stops paying. This is the same shape the earlier TTQ work found (the Q6_K → Q5_K_M cliff, then a flat top), now quantified on this model.

Note the recipe: the Q8_0 rung is the only one not shaped by the TTQ body map. It is pure Q8_0 on the 432 quantizable body tensors (301 stay F32: norms, ssm_a, ssm_dt, ssm_conv1d, routers), with the TTQ MTP map layered on top. No importance matrix was passed — Q8_0 treats every tensor identically, so a matrix has nothing to choose.

---

Vision projector

Groot is a vision-language model, but llama.cpp's GGUF path for qwen35moe keeps the vision tower out of the model file — upstream ships it separately, and so do we.

It is 11.0 BPW rather than 8.5 because 224 of the 334 tensors are F32 (norms and the merger) and stay F32, while 27 fall back to F16 because their dims are not Q8_0-block divisible.

---

Serving

llama-server -m Groot-19B-A3B-TTQ-4.70bpw-IQ4_NL.gguf \
  -ngl 99 -ncmoe 40 --ctx-size 32768 --jinja

• --n-cpu-moe N keeps the MoE weights of the first N layers on the CPU.
• MTP drafting: --spec-type draft-mtp --spec-draft-n-max 2 — 2–3 tokens ahead; deeper is not better, because later drafts are mostly rejected.
• Vision: --mmproj mmproj-Groot-19B-A3B-261023-Q8_0.gguf --image <file> on a build with qwen35moe vision support, plus --no-mmproj-offload if you want the encoder on CPU to keep more layers on the GPU or more headroom.

Sampling

Use the presets the base model publishes — they are the model's own recommendations, not ours:

Flags are llama.cpp's names for the equivalent parameters, and framework support for individual samplers varies. All KLD and top-1 figures in this card are sampler-independent — deterministic passes, not generations — so no measurement here depends on the choice.

---

How these were made

1. BF16 source. convert_hf_to_gguf.py --outtype bf16 from the safetensors release (llama.cpp a37cc461), 38,006,441,824 bytes, 753 tensors. Verified three ways: every non-GDN tensor byte-identical to the source payload, F32 tensors exact upcasts, the GDN grouped→tiled V-head reorder (PR #19468) reproduced exactly; norms stored w+1; functional check PPL 3.2520 on the eval corpus.
2. Importance matrix. Built from our frozen agentic calibration split — 108 documents, 464 chunks of 512 tokens, the eight content-type slices held within 0.4 pp of their weights (code .38 / tool_output .15 / reasoning .13 / docs .11 / diff .06 / terminal .07 / config .05 / meta .05). llama-imatrix --parse-special --no-ppl, KV pinned f16. It covers 510 of the 753 tensors — every expert, shared-expert, attention/GDN projection and router; nothing on token_embd, output, blk.40 or the norms.

Measurement protocol

Same harness for every row — the one every TTQ ladder in this account is measured with — on the same 8 GB VRAM / 32 GB RAM box.

• KLD / top-1 / PPL: divergence against a stored reference produced by the BF16 file itself, 40 chunks of 512 tokens, KV pinned to f16, a single greedy deterministic pass. ± is the standard error over chunks.
• Null test: the BF16 source against its own reference returns Mean KLD −0.000000 ± 0.000000, same-top-p 99.990%. The harness contributes nothing.
• The code corpus is the frozen agentic eval split's code/diff/terminal/config documents, reduced to payload (system preambles and reasoning blocks stripped; tool-call and tool-response wrappers kept), interleaved 7:1:1:1 so the measured window carries the mix rather than being 100% code. 26,892 tokens.
• The prose corpus is wikitext-2 test, 40 × 512 — the same protocol as our other ladders, which keeps the wikitext column comparable across our repos (the code column is not: the corpus and its mix differ).

---

Benchmarks

Greedy, single pass, same harness and packs as our other TTQ ladders. The rung column is a header because this is the only rung we benchmark; more rows land as runs finish, as a new commit rather than a new model — and all three TTQ cards are updated together when a result lands.

The Qwen TTQ 3.93 and Cyber-Tiel 3.92 columns are context, not controls — different 35B fine-tunes at the same ladder position, measured by the same harness on the same packs. Sizes are in the headers for that reason: this rung is 10.39 GiB against their 15.85 and 16.22 — 5.8 GiB smaller than Cyber-Tiel yet within 0.2 points of it on HumanEval and 2 points on GPQA.

HumanEval 98.2 (161 of 164) is a "not broken" result, not a ranking one. 164 short Python problems sit close to ceiling for models of this class — Cyber-Tiel scores the same 98.2 — and ±1.0 SE at n=164 means the two are indistinguishable. What it establishes is that the 4.70 bpw rung still writes working functions on the standard set; LiveCodeBench is the unsaturated code benchmark and will say more when it runs.

IFEval 92 on 541 prompts holds instruction following at 4.70 bpw — within a point of Qwen's 93.05 on the same pack, and ±1.2 SE at that n makes the gap noise. This is the row that would move first if quantization had cost the model its format compliance, so it is the useful negative result.

LiveCodeBench 84.3 is the one row where the sign is against us — 1,055 problems, no saturation to hide behind, and the base published 85.69. That is −1.4 points, ≈1.2 σ: not significant on its own, but it is the largest gap in this table and the first that is both unsaturated and comparable, so it is where a small real quantization loss would show. Contrast Cyber-Tiel at 86.5 — 2.2 points higher from a file 5.8 GiB larger, which is the trade this ladder is making: at 4.70 bpw you spend a little competitive-programming accuracy to save most of the size.

The reference column is the base model's own published number, which is what makes the GPQA row readable: 86 against 85.35 is +0.65, ≈0.2 σ — no measurable degradation at 4.70 bpw. The base figure is thinking mode at default reasoning effort with temp=1.0, top_p=0.95, top_k=20; ours is a single greedy pass, the stricter protocol of the two. At n=198 only a >5-point collapse resolves in one run, so read it as "not damaged", not "improved".

The base card publishes sixteen benchmarks (Terminal-Bench 2.1 52.10, SWE-bench Pro 49.79, Toolathlon Verified 42.90, NL2Repo 35.65, ALE 29.59, SWE-bench Verified 31.66, LiveCodeBench v6 85.69, GPQA 85.35, HLE 21.21 / 32.39 with tools, LongBench v2 62.03, BFCL v4 74.13, plus AutomationBench, JobBench, DeepSWE, BigCodeBench Hard). We do not re-run that suite per rung — it is agentic and hours per cell, while KLD resolves quantization damage far more sharply. The rows we do run are set against those published figures wherever the protocol allows.

---

Limits and things that did not work

• The vision tower is not in the model files. Every rung is a text GGUF. Images need this repo's mmproj and a build with qwen35moe vision support.
• Q8_0 does not pay (see above): +24% size for −11% code KLD.
• IQ3_XXS is dominated by Q3_K_S: 0.64 GiB smaller, but worse on both corpora and slower on CPU (codebook tables). It exists to complete the ladder, not to be recommended.

---

Credits and licence

Base model GrootTeam/Groot-19B-A3B-261023 (Apache-2.0), which also provides the F16 projector this repo's mmproj is quantized from.

The ladder is our own design — TokenTinkerer's. It is the same TTQ recipe we built for our Qwen3.6-35B-A3B TTQ ladder and then replayed onto our Cyber-Tiel TTQ ladder; Groot is the third model it has been applied to, unchanged. Cyber-Tiel is where the recipe was previously used, not where it came from. The BF16 conversion, importance matrix, quantization, measurements and this card are all TokenTinkerer's work.

Licence inherited from the base model: Apache-2.0. The GGUF ladder is text-only; the mmproj is a quantized derivative of upstream's F16 projector.