Laguna-XS-2.1-DFlash (ik_llama.cpp GGUF)

A GGUF conversion of poolside/Laguna-XS-2.1-DFlash, the DFlash speculative drafter for poolside/Laguna-XS-2.1. It is published so the Laguna DFlash support proposed for ik_llama.cpp can be tested without re-running the conversion.

This is not a standalone model. It only works as a DFlash draft passed with --model-draft alongside the Laguna XS 2.1 target. Loading it on its own produces nothing useful.

It requires the Laguna DFlash branch of ik_llama.cpp (the arch is not in mainline yet). The pull request adds the converter, loader, and graph support for DFlashLagunaForCausalLM; a stock build will not load this file.

Files

Matching target

Pair it with a GGUF of the target model, poolside/Laguna-XS-2.1, converted with the same branch. The runtime numbers below used a Q4_K_M target.

Usage

llama-cli -m Laguna-XS-2.1-Q4_K_M.gguf \
  -md Laguna-XS-2.1-DFlash-ik_llama-Q8_0.gguf \
  --spec-type dflash:n_max=2,cross_ctx=512 \
  -ngl 999 -ot exps=CPU -ngld 999 --jinja \
  -p "Write a quick sort implementation in python."

The proposal depth that pays off depends on where the target runs. n_max=2 is a reasonable default across setups. With the target fully on CPU the optimum is higher (n_max=4); with the target offloaded to CUDA it is lower (n_max=1-2), and the draft should also sit on GPU (-ngld 999), because a CPU-side draft stalls the fast target and loses at every depth.

The speedup is largest on structured output (code, tables) where the drafter predicts well, and can turn into a slowdown on free prose where it does not. Measure on your own workload before committing to a setting.

Validation

Conversion (deterministic):

Runtime smoke, Q8_0 draft with a Q4_K_M target, greedy, RTX 4070 + Core i7-11700K. Token generation, speculative vs non-speculative on the same prompt:

Output is coherent and correct: generated code compiles and prose stays accurate. It is not guaranteed bit-identical to non-speculative decoding on longer greedy runs, because batched block verification occasionally flips a near-tie argmax in the target. This is a property of the target model under batched evaluation, not of the draft; generic (non-Laguna) DFlash drafts are unaffected.

This is a functional evaluation with a quantized pair, not a substitute for poolside's own BF16 chat-template evaluation.

Conversion notes

• Draft source: poolside/Laguna-XS-2.1-DFlash
• Target source: poolside/Laguna-XS-2.1
• Converted with the DFlashLagunaForCausalLM path added in the ik_llama.cpp Laguna DFlash pull request. The converter splits the packed QKV, writes the per-slice auxiliary RMS norms and per-layer attention gates, and records the causal all-SWA contract that the loader and graph read back.

License

The source model is released under the OpenMDW-1.1 license, included here as LICENSE.md. That license is retained with this distribution as it requires.