Qwen3-8B-ES

Qwen3-8B-ES is a native reasoning model fine-tuned from Qwen/Qwen3-8B-Base to reason in Spanish. This model produces its entire reasoning trace in Spanish before delivering the final answer in Spanish.

It is released alongside the paper Rethinking the Multilingual Reasoning Gap with Layer Swap.

Model details

Base model: Qwen/Qwen3-8B-Base
Language: Spanish (CoT and answer)
Training: Full SFT, ~10B tokens, 2 epochs
Context length: 32,768 tokens
Dataset: lightonai/Dolci-Think-SFT-32B-Multilingual (Spanish split).

[!NOTE] The model was trained on data derived from allenai/Dolci-Think-SFT-32B, released under the ODC-BY-1.0 license.

Related models

This model is part of a Spanish specialist trio designed to study the native reasoning gap:

Model	CoT language	Description
`lightonai/Qwen3-8B-ES`	Spanish	Native reasoning specialist
`lightonai/Qwen3-8B-ES-Swap`	Spanish	Layer Swap: middle layers (L13–L22) of `Qwen3-8B-EN` transplanted into `Qwen3-8B-ES`
`lightonai/Qwen3-8B-ES-Pivot-EN`	English	Same Spanish Q&A pairs, but CoT in English
`lightonai/Qwen3-8B-EN`	English	English specialist

Evaluation

All scores are mean accuracy (%) on the Spanish version of each benchmark, with sample standard deviation across runs. AIME 24/25 is averaged over 30 runs; the others over 10 runs, using the recommended generation parameters.

Model	MGSM-Rev2	Global-MMLU-Lite	GPQA-Diamond	AIME 24/25	HumanEvalPlus	Average
`Qwen3-8B-ES`	93.20	76.58	55.15	56.11	81.00	72.41
`Qwen3-8B-ES-Swap`	97.08	77.10	55.15	58.50	86.19	74.80
`Qwen3-8B-ES-Pivot-EN`	95.08	78.20	56.57	61.33	80.44	74.32
`Qwen3-8B-EN`	94.76	78.55	54.44	61.06	83.25	74.41

Benchmarks used:

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "lightonai/Qwen3-8B-ES"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "Resuelve: 24 × 17 = ?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)

outputs = model.generate(inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, top_k=20)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Recommended sampling: temperature=1.0, top_p=0.95, top_k=20, min_p=0.

Citation

If you find our work helpful, feel free to give us a cite.

@misc{lasbordes2026rethinking,
  title        = {Rethinking the Multilingual Reasoning Gap with Layer Swap},
  author       = {Lasbordes, Maxence and Chatelain, Amélie and Seddah, Djamé},
  year         = {2026},
  eprint       = {2605.26735},
  archivePrefix= {arXiv},
  primaryClass = {cs.CL}
}