NorMistral-11B-long GGUF

GGUF builds of ltg/normistral-11b-long (Apache-2.0), for llama.cpp and anything built on it. NorMistral is an 11.4B base model for Norwegian (Bokmål and Nynorsk), Northern Sámi and the other Nordic languages, with English and code. Its context is extended to 32,768 tokens, and it uses its own 51,200-token tokenizer. There are eight sizes, from 22.9 GB down to 4.3 GB.

This is an independent conversion. It is not made or endorsed by LTG, NORA.LLM or the llama.cpp project.

This is a base model, not a chat model. It continues text; it does not follow instructions or hold a conversation, and it has no chat template. Give it the start of a document, or a few-shot pattern such as Engelsk: ...\nBokmål:, and let it complete. In the authors' words: "This model is pretrained on raw (mostly web-based) textual data. It is not finetuned to follow instructions, and it can generate harmful completions after inappropriate user prompts. It is primarily intended for research purposes."

Which file should I download?

The two right-hand columns are measured against the BF16 GGUF over ~8.7k held-out tokens of Bokmål, Nynorsk, Northern Sámi and English (see "How we checked"). Lower KL means closer to the original. Every file runs CPU-only. With a GPU, put as many layers on it as fit (-ngl), and llama.cpp keeps the rest in RAM.

Our pick:

• Q5_K_M, if you have 12 GB of RAM or more.
• Q4_K_M, on 8–12 GB machines.
• Q8_0, if you want it practically identical to the original.

Q3_K_M and Q2_K are for very small machines; expect visibly weaker text, Q2_K especially.

Context and memory. The KV cache costs about 160 KB per token at f16, so the full 32K context adds about 5.2 GB on top of the file (about 2.6 GB with -ctk q8_0 -ctv q8_0). The "fits in about" column assumes 4K.

Run it

llama.cpp (plain completion), current upstream; no patch needed:

llama-completion -m NorMistral-11B-long-Q5_K_M.gguf -no-cnv -n 64 --temp 0 \
  -p $'Engelsk: I\'m excited to try this new Norwegian language model!\nBokmål:'

llama.cpp server (raw completions endpoint, full 32K context):

llama-server -m NorMistral-11B-long-Q5_K_M.gguf -c 32768
curl http://127.0.0.1:8080/completion -d '{"prompt": "Hurtigruten er en norsk skipsrute som", "n_predict": 64}'

Use /completion (or /v1/completions), not the chat endpoint: the model has no chat template.

Translation pattern from the authors. Prompt Engelsk: {text}\nBokmål: and stop at the first newline. The same pattern works for Nynorsk and Northern Sámi targets (see the smoke tests below).

How we made it

1. The upstream tokenizer_config.json names token ids 5–8 <special_0>–<special_3>, while tokenizer.json names them <instruction>, </instruction>, <system_prompt>, </system_prompt>. transformers then appends four phantom ids past the 51,200-row embedding, and the converter refuses. We converted from a copy whose tokenizer_config.json follows tokenizer.json. No weights were modified. Normal text never touches these ids.
2. The tokenizer's pre-tokenizer regex is the same as Mistral-Nemo's ("tekken"), so the GGUF uses llama.cpp's existing tekken pre-tokenizer.
3. We converted to BF16 with llama.cpp's convert_hf_to_gguf.py, then quantized from that BF16.
   • Every quant except Q8_0 uses an importance matrix computed on the BF16 over 129k tokens (252 × 512).
   • The imatrix text is a general calibration text (78k tokens) plus a sample of Wikipedia: 40 Bokmål, 10 Nynorsk and 10 Northern Sámi articles (51k tokens; CC BY-SA 4.0; random articles of 1,500+ characters, fetched through the Wikipedia API on 2026-10-06; titles are listed in our build notes). An English-only imatrix would under-serve a Nordic model.

How we checked

• Tokenizer. We compared llama.cpp's tokenizer with the HF tokenizer token-for-token. Seven hand-written strings in Bokmål, Nynorsk, Northern Sámi (with á č đ ŋ š ŧ ž) and English all match. So do 1,253 of 1,255 paragraphs (139k tokens) of the imatrix and held-out texts. The two misses are one word, EØS. llama.cpp's tekken regex approximates Unicode upper/lower-case classes, so an all-caps word that contains Æ, Ø or Å can split differently. The text is the same, only the token boundaries differ. Mistral-Nemo GGUFs share this behaviour.

• Conversion vs the original. We ran the HF checkpoint in float32 on CPU over a 161-token paragraph in Bokmål with two sentences of Northern Sámi (plain text, no template). We compared logits with the BF16 GGUF: mean KL 2.4e-5, top-1 agreement 159/161. Both disagreements are near-exact ties in the original: its own logit margin between the two candidates is 0.0003 and 0.0077. llama.cpp's bf16 arithmetic tips them the other way. This is rounding, not a conversion error. The GGUF computes what the original computes.

  File	top-1 vs HF fp32	mean KL vs HF fp32
  BF16	159/161	0.00002
  Q8_0	155/161	0.0008
  Q6_K	154/161	0.0059
  Q5_K_M	153/161	0.0085
  Q4_K_M	147/161	0.0190
  IQ4_XS	140/161	0.0223
  Q3_K_M	136/161	0.0541
  Q2_K	125/161	0.1865

• Quants vs BF16. We ran llama-perplexity --kl-divergence over 17 × 512 tokens of held-out text. It holds 12 Bokmål, 6 Nynorsk and 4 Northern Sámi Wikipedia articles, none of them in the imatrix sample, plus English technical notes of our own. Those are the numbers in the first table. BF16 perplexity on this text is 4.44.

• Long context. We hid three access codes at 10%, 50% and 90% depth in real Norwegian and English text and asked for each by plain-text completion (Spørsmål: ... Svar: Tilgangskoden til ... er).

  • At ~8K tokens, Q4_K_M returned 3/3.
  • At ~30K tokens, both Q4_K_M and Q8_0 returned 1/3, with identical answers. The model did read across the whole window: two codes it gave came from 50% and 90% depth. But it bound them to the wrong place, and it never returned the code at 10% depth.
  • Q8_0 and Q4_K_M agree exactly, so this is how the model itself behaves on this test, not quantization damage.
  • The 32K window loads and runs; just don't expect reliable recall of facts near its far end.

• Smoke tests (Q4_K_M, greedy, first line of each completion):

  • Engelsk: I'm excited to try this new Norwegian language model!\nBokmål: → Jeg er spent på å prøve denne nye norske språkmodellen!
  • Bokmål: Jeg bor i en liten bygd ved fjorden, og om høsten går vi ofte på tur i fjellet.\nNynorsk: → Eg bur i ei lita bygd ved fjorden, og om hausten går vi ofte på tur i fjellet.
  • Norsk: Jeg bor i Kautokeino og snakker samisk.\nNordsamisk: → Mun orun Guovdageainnus ja human sámegiela.
  • Hurtigruten er en norsk skipsrute som → går langs norskekysten fra Bergen til Kirkenes. …
  • Sámegiella lea → virggálaš giella sihke sápmelaččain ja almmolaš hálddahusas …
  • The three largest cities in Norway are → Oslo, Bergen, and Trondheim. …

Not tested:

• downstream benchmark accuracy;
• contexts beyond the long-context check above;
• GPU backends (CUDA, Vulkan, Metal); every check above ran on CPU;
• Swedish, Danish, Icelandic and Faroese prompts (the model was trained on them; we did not check them).

Please report anything odd in the Community tab.

Licence

Apache-2.0, same as the original. The authors say: "We release the model under Apache 2.0 license to indicate that we do not impose any additional constraints on the model weights. However, we do not own the data in the training collection." The training collection is described on the original model card.

All credit for the model goes to the Language Technology Group at the University of Oslo (LTG) and NORA.LLM. Please cite their paper:

@inproceedings{samuel-etal-2025-small,
    title = "Small Languages, Big Models: {A} Study of Continual Training on Languages of {Norway}",
    author = "Samuel, David and Mikhailov, Vladislav and Velldal, Erik and {\O}vrelid, Lilja and
              Charpentier, Lucas Georges Gabriel and Kutuzov, Andrey and Oepen, Stephan",
    booktitle = "Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)",
    year = "2025",
    address = "Tallinn, Estonia",
    publisher = "University of Tartu Library",
    url = "https://aclanthology.org/2025.nodalida-1.61/",
    pages = "573--608",
}

Made by Hob-forge: verified quants of models other people can't run yet.