[!IMPORTANT] This is the FP8 (dynamic) quantized version of Latxa 3, our largest and most capable Basque model to date.
Latxa 3 is a Basque-adapted, multimodal instruct model built on top of Qwen3.5-122B-A10B, a Mixture-of-Experts model with 122B total and ~10B active parameters. It follows instructions and holds conversations in Basque, understands images, and reasons (thinks) before answering. Only the thinking mode is supported.
You can try Latxa 3 at latxa.hitz.eus.
Latxa is a family of Large Language Models (LLMs) for Basque. Languages with few resources get much less attention from model developers than English, and models trained mostly on English data perform poorly in them. Latxa models narrow this gap for Basque. Latxa 3 has been adapted following the methodology described in Sainz et al. (2025), on top of Qwen3.5-122B-A10B.
The weights are quantized to FP8 with llm-compressor (compressed-tensors format, FP8_DYNAMIC scheme):
lm_head, embeddings, MoE router gates, linear-attention conv1d/in_proj_a/in_proj_b, the vision encoder, and the MTP projection.This roughly halves memory use compared to BF16 (~125GB of weights). The model runs on 2×A100 80GB GPUs (our setup) or on a single H200 (141GB). See Getting Started.
We recommend serving the model with vLLM, which supports the compressed-tensors FP8 format natively:
2×A100 80GB (the setup we use):
vllm serve HiTZ/Latxa-3-122B-A10B-FP8 \
--served-model-name Latxa-3 \
--tensor-parallel-size 2 \
--enable-expert-parallel \
--max-model-len 262144 \
--gpu-memory-utilization 0.95 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--reasoning-parser qwen3
[!NOTE] A100 GPUs have no native FP8 support, so vLLM uses weight-only FP8 kernels (Marlin) on them. Hopper GPUs (H100/H200) and newer run full FP8 (W8A8) and will be faster.
Single H200 (141GB):
vllm serve HiTZ/Latxa-3-122B-A10B-FP8 \
--served-model-name Latxa-3 \
--max-model-len 131072 \
--gpu-memory-utilization 0.95 \
--kv-cache-dtype fp8 \
--max-num-seqs 16 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--reasoning-parser qwen3
On a single H200 the weights take up most of the memory, which leaves only a few GB for the KV cache. Only 12 of the 48 layers use full attention, so the cache is small (~12KB per token with an FP8 KV cache). Still, we recommend limiting the context length and the number of concurrent sequences. With low concurrency you can raise --max-model-len to the full 262144 tokens. If you hit OOM errors at startup, lower --max-num-seqs or --max-model-len.
Then query it with any OpenAI-compatible client:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
messages = [
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}},
{"type": "text", "text": "Zer ikusten da irudi honetan?"},
],
},
]
response = client.chat.completions.create(
model="Latxa-3",
messages=messages,
temperature=0.5,
top_p=0.9,
presence_penalty=0.0,
max_tokens=65536,
extra_body={"top_k": 20, "min_p": 0.0, "repetition_penalty": 1.0},
)
print(response.choices[0].message.content)
It can also be used with the transformers pipeline:
from transformers import pipeline
pipe = pipeline("image-text-to-text", model="HiTZ/Latxa-3-122B-A10B-FP8", device_map="auto")
messages = [
{
"role": "user",
"content": [
{"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
{"type": "text", "text": "Zer ikusten da irudi honetan?"},
],
},
]
print(pipe(text=messages, max_new_tokens=1024))
[!TIP] We recommend the following sampling parameters:
Parameter Value temperature 0.5 top_p 0.9 top_k 20 min_p 0.0 presence_penalty 0.0 repetition_penalty 1.0 max_tokens 65536
max_tokenscan be increased if needed (e.g. for long reasoning traces). Support for these parameters varies by inference framework.
[!WARNING] Latxa 3 currently supports thinking mode only. Unlike the base Qwen3.5 model, it has not been trained to answer without reasoning. Do not disable thinking (e.g. with
enable_thinking: Falsein the chat template), because output quality is not guaranteed. With--reasoning-parser qwen3, vLLM returns the reasoning trace separately, in thereasoning_contentfield.
Latxa models are intended to be used with Basque data. Performance in other languages is not guaranteed.
Latxa 3 is instruction-tuned and can follow instructions or work as a chat assistant, including on tasks that involve images.
The model is not intended for malicious activities, such as harming others or violating human rights. Any downstream application must comply with current laws and regulations. Irresponsible use in production environments without proper risk assessment and mitigation is also discouraged.
To reduce potentially disturbing or harmful content, Latxa has been trained on carefully selected and processed data, which comes mainly from local media, national/regional newspapers, encyclopedias and blogs (see Latxa Corpus v2). Still, the model is based on Qwen3.5 and may carry the same biases, risks and limitations.
As with any quantized model, the FP8 version may behave slightly differently from the original BF16 weights.
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
Latxa 3 was trained on Latxa Corpus v2, following the adaptation recipe of Sainz et al. (2025). Before training the large models, we ran development experiments on smaller Qwen3.5 models. These compared training configurations and tested whether to add reasoning traces to the instruction data and whether to include multimodal data. The best configuration was then used to train three Qwen3.5 models (27B, 35B-A3B and 122B-A10B). The 122B-A10B model performed best, so it received an additional training pass. That model is Latxa 3.
The model was trained with Swift/Megatron on the MareNostrum 5 Accelerated Partition (BSC). Jobs ran on 32 and 64 nodes (128 and 256 NVIDIA H100 64GB GPUs).
We compare Latxa 3 with its base model, Qwen3.5-122B-A10B. As a reference, we also include Latxa 2 (Llama-3.1-70B), adapted with the same recipe. The largest gains are in Basque language proficiency (+12.4 points) and knowledge about Basque culture (+21.2 points). General knowledge stays the same or improves slightly.
All datasets are evaluated as text generation: the model produces an answer, and that answer is scored. Multiple-choice tasks are scored by checking the chosen option. Open-ended questions (marked open) are scored by an LLM judge, which compares the model's answer with a reference answer.
[!NOTE] These results were obtained with the BF16 weights.
Basque language proficiency (accuracy, %)
| Task | Latxa 2 (Llama-3.1-70B) | Qwen3.5-122B-A10B | Latxa 3 |
|---|---|---|---|
| EusProficiency | 70.8 | 61.1 | 82.4 |
| EusReading | 80.4 | 84.4 | 87.8 |
| BAT (Batua norms) | 75.9 | 69.3 | 81.8 |
| Average | 75.7 | 71.6 | 84.0 |
Basque knowledge (accuracy, %)
| Task | Latxa 2 (Llama-3.1-70B) | Qwen3.5-122B-A10B | Latxa 3 |
|---|---|---|---|
| EusExams v2 | 76.2 | 81.7 | 84.9 |
| EusTrivia | 84.1 | 77.8 | 87.8 |
| EusTrivia (open) | 59.9 | 47.6 | 63.5 |
| MMLU-Pro-eus | 54.0 | 76.8 | 77.2 |
| MMLU-Pro-eus (open) | 42.3 | 67.8 | 68.5 |
| Average | 63.3 | 70.3 | 76.4 |
Cultural knowledge (accuracy, %). Local questions are about Basque culture. Global questions are about general knowledge and serve as a control.
| Task | Latxa 2 (Llama-3.1-70B) | Qwen3.5-122B-A10B | Latxa 3 |
|---|---|---|---|
| Local, multiple choice, Basque | 81.7 | 69.7 | 85.8 |
| Local, multiple choice, English | 77.8 | 66.9 | 83.6 |
| Local, open-ended, Basque | 62.5 | 31.4 | 60.3 |
| Local, open-ended, English | 52.4 | 28.0 | 51.1 |
| Local average | 68.6 | 49.0 | 70.2 |
| Global, multiple choice, Basque | 91.9 | 93.6 | 94.4 |
| Global, multiple choice, English | 92.2 | 93.6 | 93.3 |
| Global, open-ended, Basque | 73.2 | 74.9 | 77.2 |
| Global, open-ended, English | 74.4 | 80.2 | 78.5 |
| Global average | 82.9 | 85.6 | 85.9 |
Machine translation (chrF). Sentence level uses FLORES and document level uses ALIA DocMT.
| Direction | Latxa 2 (Llama-3.1-70B) | Qwen3.5-122B-A10B | Latxa 3 |
|---|---|---|---|
| FLORES en → eu | 60.5 | 57.7 | 60.2 |
| FLORES es → eu | 54.7 | 52.3 | 54.5 |
| FLORES into Basque (avg.) | 57.6 | 55.0 | 57.3 |
| FLORES eu → en | 62.2 | 60.5 | 61.4 |
| FLORES eu → es | 50.7 | 50.5 | 51.1 |
| FLORES out of Basque (avg.) | 56.5 | 55.5 | 56.3 |
| DocMT en → eu | 62.0 | 62.3 | 65.6 |
| DocMT es → eu | 70.2 | 66.5 | 73.4 |
| DocMT into Basque (avg.) | 66.1 | 64.4 | 69.5 |
| DocMT eu → en | 67.0 | 66.3 | 67.3 |
| DocMT eu → es | 68.4 | 71.9 | 73.6 |
| DocMT out of Basque (avg.) | 67.7 | 69.1 | 70.5 |
In all tables, bold marks the best score in each row.
[!WARNING] The paper describing this model is in preparation. Further training and evaluation details will be released with it.
Energy use is estimated from GPU hours, assuming an average draw of 0.7 kW per H100 GPU (its nominal TDP) and a grid emission factor of 0.155 kg CO2eq/kWh.
@inproceedings{sainz-etal-2025-instructing,
title = "Instructing Large Language Models for Low-Resource Languages: A Systematic Study for {B}asque",
author = "Sainz, Oscar and
Perez, Naiara and
Etxaniz, Julen and
Fernandez de Landa, Joseba and
Aldabe, Itziar and
Garc{\'i}a-Ferrero, Iker and
Zabala, Aimar and
Azurmendi, Ekhi and
Rigau, German and
Agirre, Eneko and
Artetxe, Mikel and
Soroa, Aitor",
booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
year = "2025",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.emnlp-main.1484/",
doi = "10.18653/v1/2025.emnlp-main.1484",
pages = "29124--29148"
}
This work has been partially supported by the Basque Government (Research group funding IT1570-22 and IKER-GAITU project). Latxa 3 was trained on the MareNostrum 5 supercomputer at the Barcelona Supercomputing Center (BSC), through the EuroHPC Joint Undertaking AI Factory Large Scale Access project EHPC-AIF-2026LS03-009. EJIE (Basque Government Informatic Society) led that project together with HiTZ (UPV/EHU). We thank the BSC support team for their help in setting up the software environment.
model-00001-of-00061.safetensors 2.5 GBmodel-00018-of-00061.safetensors 2.5 GBmodel-00023-of-00061.safetensors 2.5 GBmodel-00028-of-00061.safetensors 2.5 GBmodel-00033-of-00061.safetensors 2.5 GBmodel-00038-of-00061.safetensors 2.5 GBmodel-00043-of-00061.safetensors 2.5 GBmodel-00048-of-00061.safetensors 2.5 GBmodel-00053-of-00061.safetensors 2.5 GBmodel-00058-of-00061.safetensors 2.5 GBmodel-00013-of-00061.safetensors 2.5 GBmodel-00003-of-00061.safetensors 2.5 GBmodel-00008-of-00061.safetensors 2.5 GBmodel-00017-of-00061.safetensors 2.5 GBmodel-00022-of-00061.safetensors 2.5 GBmodel-00027-of-00061.safetensors 2.5 GBmodel-00032-of-00061.safetensors 2.5 GBmodel-00037-of-00061.safetensors 2.5 GBmodel-00042-of-00061.safetensors 2.5 GBmodel-00047-of-00061.safetensors 2.5 GBmodel-00052-of-00061.safetensors 2.5 GBmodel-00057-of-00061.safetensors 2.5 GBmodel-00002-of-00061.safetensors 2.5 GBmodel-00007-of-00061.safetensors 2.5 GBmodel-00012-of-00061.safetensors 2.5 GBmodel-00014-of-00061.safetensors 2.5 GBmodel-00019-of-00061.safetensors 2.5 GBmodel-00024-of-00061.safetensors 2.5 GBmodel-00029-of-00061.safetensors 2.5 GBmodel-00034-of-00061.safetensors 2.5 GBmodel-00039-of-00061.safetensors 2.5 GBmodel-00044-of-00061.safetensors 2.5 GBmodel-00049-of-00061.safetensors 2.5 GBmodel-00054-of-00061.safetensors 2.5 GBmodel-00059-of-00061.safetensors 2.5 GBmodel-00004-of-00061.safetensors 2.5 GBmodel-00009-of-00061.safetensors 2.5 GBmodel-00061-of-00061.safetensors 2.3 GBmodel-00015-of-00061.safetensors 1.7 GBmodel-00020-of-00061.safetensors 1.7 GBmodel-00025-of-00061.safetensors 1.7 GBmodel-00030-of-00061.safetensors 1.7 GBmodel-00035-of-00061.safetensors 1.7 GBmodel-00040-of-00061.safetensors 1.7 GBmodel-00045-of-00061.safetensors 1.7 GBmodel-00050-of-00061.safetensors 1.7 GBmodel-00055-of-00061.safetensors 1.7 GBmodel-00060-of-00061.safetensors 1.7 GBmodel-00005-of-00061.safetensors 1.7 GBmodel-00010-of-00061.safetensors 1.7 GBmodel-00016-of-00061.safetensors 907 MBmodel-00021-of-00061.safetensors 907 MBmodel-00026-of-00061.safetensors 907 MBmodel-00031-of-00061.safetensors 907 MBmodel-00036-of-00061.safetensors 907 MBmodel-00041-of-00061.safetensors 907 MBmodel-00046-of-00061.safetensors 907 MBmodel-00051-of-00061.safetensors 907 MBmodel-00056-of-00061.safetensors 907 MBmodel-00006-of-00061.safetensors 907 MBmodel-00011-of-00061.safetensors 907 MBHiTZ/Latxa-3-122B-A10B