Laguna-XS-2.1 — APEX GGUF

APEX (dynamic, role/band-aware bit allocation) and standard GGUF quants of poolside/Laguna-XS-2.1 — a 40-layer, 256-routed-expert (+ shared expert) MoE with alternating attention/dense blocks, real BF16 total size 62.35 GiB. This is a genuine instruct/chat model (has a working chat template with a <think> reasoning channel) — verified with a real chat completion, not just inferred from metadata.

update:
-v2 files: identical bit allocation to the originals, except the 48 attn_gate coefficients are kept at F32 instead of the tier's attention band type. They parameterise a nonlinearity that multiplies the state path, so quantising them lets error compound along the sequence; F32 costs about 31 MB. Prefer -v2.

Sizes

On the 16GB-card size point specifically: dynamic-APEX lands smaller than a stock IQ3_XXS at the same rough budget (11.25 vs 12.06 GiB) but scores very slightly worse on raw PPL (15.76 vs 15.56, +1.3% relative to each other) — a real, honestly-reported result, not a clean win for either approach at this exact size/architecture point. The value of the dynamic-APEX tier here is landing precisely on a chosen VRAM budget via role/depth-aware allocation, not a free quality win over a well-tuned stock quant.

i-quality is the tier to reach for if you're not VRAM-constrained to 16GB — it recovers to within 1% of full bf16 quality at under a third of the size.

Agentic benchmark (OpenCode 2, 42 tasks)

Two runs per tier with the -v2 files. The harness is quantkit bench/ with OpenCode 2's real agentic loop, graded by a fresh pytest. It ran on a DGX Spark (GB10) with stock llama.cpp 6d0549831, temp 1.0, top-k 20, top-p 0.95, ctx 262144 and an output cap of 32768.

A full score is the entry bar. Between tiers that clear it, the discriminating number is output tokens on the tasks every run passed.

• i-quality-v2 is the pick for agentic coding. It passed every task in both runs.
• APEX-dynamic-v2 failed the same task in both runs. On btree_insert_delete it looped until the one-hour limit, at about 100k tokens each time. i-quality-v2 solved that task on 5.7k and 38k tokens.
• Token totals on shared tasks favour i-quality-v2 by about 9% on average. That is inside the run-to-run spread at two runs, because one frontier task alone swings between 28k and 82k tokens.
• APEX-dynamic-v2 is still a capable 12 GB coder if that is the memory you have. Expect more loops, and set a wall-clock or token limit on long tasks.

Structural allocation (dynamic-APEX)

Both APEX tiers use dynamic_apex.py's structural mode: routed experts are bucketed into edge / near / middle depth-bands (edge = first/last layers, generally most load-bearing) and allocated decreasing precision toward the middle:

• i-quality (19.18 GiB, 4.59 bpw): routed edge → Q6_K, near → Q4_K, middle → Q3_K.
• APEX-dynamic (11.25 GiB, ~2.87 bpw): routed edge → Q3_K, near → IQ3_XXS, middle → IQ2_XXS.

Attention, shared-expert, and router tensors stay at higher fixed precision (Q6_K–Q8_0) in both tiers.

Calibration

Single imatrix (laguna-xs.imatrix) generated with llama-imatrix over a diverse, code-weighted local corpus, used identically for every quant in this repo — only the bit-allocation differs between tiers, isolating that as the sole variable in the size/PPL comparison above.

Files

• laguna-xs.imatrix — the imatrix used for every quant below.
• Laguna-XS-2.1-APEX-i-quality.gguf (19.18 GiB) — recommended default; best quality-per-byte of the tiers here.
• Laguna-XS-2.1-APEX-dynamic.gguf (11.25 GiB) — role-aware allocation targeting a 16GB-card budget.
• Laguna-XS-2.1-IQ3_XXS.gguf (12.06 GiB) — stock imatrix-guided baseline, included for the size/PPL comparison above (not the recommended pick — i-quality or APEX-dynamic both do better per byte).

Attribution

• Base model: Poolside — poolside/Laguna-XS-2.1.
• APEX recipe & toolkit: LocalAI — localai-org/apex-quant.
• Quantization engine: llama.cpp (ggml-org).

Unofficial community quantization; not affiliated with or endorsed by Poolside.