NorMistral-11B-thinking GGUF (imatrix)

Importance-matrix GGUF builds of ltg/normistral-11b-thinking (Apache-2.0), for llama.cpp and anything built on it. NorMistral-11B-thinking is a Norwegian (Bokmål and Nynorsk) chat model that reasons before it answers. It was post-trained from NorMistral-11B-long. There are four sizes, from 6.9 GB down to 4.3 GB.

This is an independent conversion. It is not made or endorsed by LTG, NORA.LLM or the llama.cpp project.

How this differs from the authors' own GGUFs. The authors publish ltg/normistral-11b-thinking-gguf: BF16, F16, Q8_0, Q6_K, Q5_K_M, Q5_0 and Q4_K_M. Use those for 5-bit and up. We publish only what they don't, plus one file that measurably improves on theirs:

• the smaller sizes IQ4_XS, Q3_K_M and Q2_K, for 8 GB machines;
• an importance-matrix Q4_K_M, calibrated partly on Norwegian and chat-formatted text. On our held-out test its mean KL vs BF16 is 0.041, against 0.052 for ltg's Q4_K_M: about 21% lower (± 0.0015 and ± 0.0018). The share of same top tokens is a tie within noise (91.8% vs 91.3%);
• every file comes with measured distance from the original (tables below).

Which file should I download?

The two right-hand columns are measured against the BF16 GGUF over ~8.7k held-out tokens of Bokmål, Nynorsk, Northern Sámi and English (see "How we checked"). Lower KL means closer to the original. Every file runs CPU-only. With a GPU, put as many layers on it as fit (-ngl), and llama.cpp keeps the rest in RAM.

For BF16, Q8_0, Q6_K or Q5_K_M, use ltg's own files.

Our pick:

• Q4_K_M (ours), on 8–12 GB machines.
• IQ4_XS, if Q4_K_M is a squeeze; it is almost as close to the original.

Q3_K_M and Q2_K are for very small machines. Expect weaker reasoning, Q2_K especially.

Reasoning traces are long, so leave room for context. The KV cache costs about 160 KB per token at f16: 8K of context adds about 1.3 GB.

Run it

llama.cpp server (OpenAI-compatible chat), current upstream; no patch needed:

llama-server -m NorMistral-11B-thinking-Q4_K_M.gguf --jinja -c 16384 \
  --temp 0.3 --top-k 64 --top-p 0.9 --repeat-penalty 1.0

These are the authors' recommended sampling settings. Keep the repetition penalty off; the authors say it can badly damage outputs.

llama.cpp CLI chat:

llama-cli -m NorMistral-11B-thinking-Q4_K_M.gguf --jinja -c 16384 --temp 0.3 --top-k 64 --top-p 0.9 --repeat-penalty 1.0

Reasoning. The model always writes a reasoning block, <think> ... </think>, and then its answer. The chat template has no switch to turn reasoning off. We tried prefilling an empty <think> </think> after the instruction. The model ignores it: it writes its reasoning anyway, closes </think> again and then answers, so prefilling does not skip reasoning. llama-server returns the <think> ... </think> block inside content, not as separate reasoning_content, so split on </think> yourself, as the authors' examples do.

The chat template

The embedded template is the authors' own, unchanged:

<s><system_prompt> {system}</system_prompt><instruction> {user}</instruction> {assistant}</s><instruction> {user}</instruction>

The model then writes <think> {reasoning}</think> {answer}</s>. Earlier assistant turns are sent without their reasoning, as in training.

How we made it

1. The upstream config.json says max_position_embeddings: 1024000. The model was post-trained from NorMistral-11B-long, which is trained to 32,768 tokens, and its tokenizer config says model_max_length: 32768. We converted with the context length set to 32,768, so that llama.cpp's default context is the trained one instead of a ~160 GB KV cache. No weights were modified.
2. The tokenizer's pre-tokenizer regex is the same as Mistral-Nemo's ("tekken"), so the GGUF uses llama.cpp's existing tekken pre-tokenizer.
3. We converted to BF16 with llama.cpp's convert_hf_to_gguf.py, then quantized from that BF16. Every quant uses an importance matrix computed on the BF16 over 184k tokens (359 × 512). The text was:
   • our NorMistral-11B-long recipe: a general calibration text plus 40 Bokmål, 10 Nynorsk and 10 Northern Sámi Wikipedia articles (CC BY-SA 4.0);
   • plus, rendered with the model's own chat template and with special tokens parsed, 12 SFT conversations and 400 Bokmål/Nynorsk user prompts from the authors' published training data.

How we checked

• Tokenizer. We compared llama.cpp's tokenizer with the HF tokenizer token-for-token. Seven hand-written strings in Bokmål, Nynorsk, Northern Sámi and English all match. So do 1,253 of 1,255 paragraphs (139k tokens). The two misses are one word, EØS. llama.cpp's tekken regex approximates Unicode case classes, so an all-caps word containing Æ, Ø or Å can split differently.

• Chat template. The template embedded in the GGUF is byte-identical to the authors' chat_template.jinja. For a two-turn conversation (system, user, assistant, user), llama.cpp's rendering tokenizes to exactly the same 35 token ids as HF apply_chat_template. llama.cpp drops the literal <s> from the rendered text and adds BOS when it tokenizes.

• Conversion vs the original. We ran the HF checkpoint in float32 on CPU over a 128-token chat: a system prompt and two turns in Bokmål and Nynorsk, including a <think> block. We compared logits with the BF16 GGUF: top-1 agreement 128/128, mean KL < 0.0001 (max 0.0004). The GGUF computes what the original computes. The same prompt against each quant:

  File	top-1 vs HF fp32	mean KL vs HF fp32
  BF16 (not published here)	128/128	< 0.0001
  Q4_K_M	125/128	0.012
  IQ4_XS	125/128	0.018
  Q3_K_M	123/128	0.035
  Q2_K	116/128	0.148

• Quants vs BF16. We ran llama-perplexity --kl-divergence over 17 × 512 held-out tokens. They are 12 Bokmål, 6 Nynorsk and 4 Northern Sámi Wikipedia articles, none in the imatrix text, plus English technical notes of our own. Those are the numbers in the first table; ltg's Q4_K_M was measured the same way. BF16 perplexity on this text is 7.52.

• Smoke tests. We ran seven real instructions on IQ4_XS and on BF16, with the authors' sampling settings and a fixed seed:

  • an explanation, a short e-mail and a reasoning question, each in Bokmål and in Nynorsk, plus one English instruction;
  • every answer on both files opened with a <think> block and ended cleanly on the end-of-sequence token;
  • the explanations and e-mails are fluent and on task, in the right written standard;
  • both checkable reasoning questions came out wrong on BF16 as well as on IQ4_XS. A train leaving 08:47 for 6 h 38 min arrives at 15:25; the model said 14:45. "17 sheep, all but 9 run away" leaves 9; the model said 8;
  • the English instruction was answered in Norwegian on both files.

  Since BF16 does the same, these are the model's own behaviour, not quantization damage. They are single samples, not a benchmark.

Not tested:

• benchmark accuracy (the authors report NorEval results on their card);
• long contexts beyond the smoke tests;
• GPU backends other than CPU.

Please report anything odd in the Community tab.

Training data (from the authors)

See ltg/normistral-11b-thinking for the full description.

1. SFT. English responses and reasoning traces generated by Kimi-K2-Thinking. The fine-tuning data therefore includes Kimi-K2 outputs, and Kimi-K2-Thinking is released under a modified MIT licence. Moonshot AI's one modification: a commercial product or service built on the software "or any derivative works thereof" with more than 100 million monthly active users, or more than US$20 million in monthly revenue, must prominently display "Kimi K2" in its user interface. Whether that clause reaches a model trained on Kimi-K2 outputs is a question for your own reading of the licence; we only point it out.
2. On-policy RL (d-RLAIF). A large collection of Norwegian Bokmål and Nynorsk prompts, with Mistral-Large-Instruct-2411 as the reward model.

Both are published at norallm/normistral-11b-thinking-training.

Licence

Apache-2.0, same as the original. The authors say: "We release the model under Apache 2.0 license to indicate that we do not impose any additional constraints on the model weights. However, we do not own the data in the training collection."

All credit for the model goes to the Language Technology Group at the University of Oslo (LTG) and NORA.LLM. Please cite their paper:

@misc{samuel2025fluentalignmentdisfluentjudges,
      title={Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languages},
      author={David Samuel and Lilja Øvrelid and Erik Velldal and Andrey Kutuzov},
      year={2025},
      eprint={2512.08777},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2512.08777},
}

Made by Hob-forge: verified quants of models other people can't run yet.