ELYZA-Thinking-1.0-llm-jp-4-32b-a3b

Highlights

ELYZA-Thinking-1.0-llm-jp-4-32b-a3b is a reasoning model by ELYZA, Inc., mid-trained and post-trained on llm-jp/llm-jp-4-32b-a3b-base for Japanese and English language understanding and generation.

• Fully domestic foundation. Built on llm-jp-4, pre-trained from scratch in Japan by LLM-jp; all further training was done by ELYZA.
• Focus on Japanese reasoning and instruction following. We localized high-quality math, coding and STEM reasoning data into Japanese and built Japanese instruction-following data with multiple checkable constraints, to improve Japanese reasoning and instruction following.
• Specialized in Japanese knowledge recall. We put particular care into Japanese-specific knowledge by synthesizing knowledge data from Japanese Wikipedia and Wikidata and training on it, to improve the model's understanding and recall of Japanese knowledge.

Average performance

Benchmarks are first averaged within 7 capability groups, then across the groups (Japanese / English: that language's benchmarks only).

MoE models

Dense models

Benchmark Results

Per-benchmark scores of the MoE models above. For the dense models, see the model card of elyza/ELYZA-Thinking-1.0-llm-jp-4-33b.

* Function calling (BFCL v4, Nejumi BFCL) excludes parallel calls for all models, as our harmony format does not support them; BFCL 10.00 / Nejumi 11.58 = floor (no calls made).
* Max output: native context − 4,096 tokens.
* Temperature 0.6 / top_p 0.95 by default (lower for JHumanEval, JFBench and function calling). LiveCodeBench / AIME / PolyMath use each model's recommended value (1.0 for gpt-oss, Qwen3.5, Gemma 4 and Nemotron 3.5).
* Reasoning effort: high for llm-jp-4 / llm-jp-4.1 and gpt-oss-20b; ELYZA-Thinking and the other models have no effort setting and run with thinking on.

Model Overview

Three training stages on top of llm-jp/llm-jp-4-32b-a3b-base, all done by ELYZA:

Mid-training: a broader foundation for knowledge, reasoning and agentic use.

• Math, coding and STEM problems were translated into Japanese and given newly generated reasoning; the original English data was kept as well.
• Agentic data was localized into Japanese while keeping tool calls, function names, arguments and JSON intact.

SFT: the answer format and Japanese ability.

• Several million synthesized SFT examples with thinking traces, about half of them high-quality Japanese data; answers were generated by a teacher model and kept only when a program verified them (math equivalence, unit tests, rule checkers).
• Complex instructions with multiple checkable constraints, including Japanese-specific ones such as furigana and kanji numerals.
• Japanese knowledge data synthesized from the latest Japanese Wikipedia and Wikidata.

Reinforcement learning (RLVR): accuracy on hard problems that imitation alone does not reach.

• Verifiable problems across math, coding, Japanese knowledge, complex instruction following and tool use, trained together in one run.
• Rewards are computed by programs: answer equivalence, test execution, constraint checkers, and call matching or the final environment state for tool use.
• Problems were selected so the model sometimes, but not reliably, solves them; reward loopholes (answers that score without solving) were found and closed before training.

See also the sibling model elyza/ELYZA-Thinking-1.0-llm-jp-4-33b.

Quickstart

vLLM (recommended)

1. Install vLLM (tested with 0.30.0) and FlashInfer's prebuilt kernels, so no local CUDA toolkit is needed:

pip install vllm==0.30.0
pip install flashinfer-jit-cache==0.6.18.post1 --extra-index-url https://flashinfer.ai/whl/cu130

2. Download the plugins:

hf download elyza/ELYZA-Thinking-1.0-llm-jp-4-32b-a3b --include "vllm_plugins/*" --local-dir .

3. Start the server:

vllm serve elyza/ELYZA-Thinking-1.0-llm-jp-4-32b-a3b \
  --max-model-len 65536 \
  --trust-remote-code \
  --moe-backend triton \
  --reasoning-parser-plugin vllm_plugins/multi_parser_loader.py \
  --reasoning-parser llmjp4 \
  --tool-parser-plugin vllm_plugins/llmjp_harmony_tool_parser.py \
  --tool-call-parser llmjp_harmony \
  --enable-auto-tool-choice

4. Send a request. The answer is returned in content, and the reasoning separately in the message's reasoning field:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "elyza/ELYZA-Thinking-1.0-llm-jp-4-32b-a3b",
    "messages": [{"role": "user", "content": "仕事の熱意を取り戻すためのアイデアを5つ挙げてください。"}],
    "temperature": 0.6,
    "top_p": 0.95
  }'

Tool calling uses the standard OpenAI tools field; calls come back in tool_calls. The bundled tool parser sets the decoding options it needs (keeping the <|call|> marker), so no extra request parameters are required:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "elyza/ELYZA-Thinking-1.0-llm-jp-4-32b-a3b",
    "messages": [{"role": "user", "content": "東京の天気を教えて"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather of a city",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
      }
    }]
  }'

Transformers

Works with the latest Transformers (tested with 4.57.6 and 5.18.0):

pip install -U transformers accelerate torch

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "elyza/ELYZA-Thinking-1.0-llm-jp-4-32b-a3b"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    dtype=torch.bfloat16,
    trust_remote_code=True,
)
model.eval()

messages = [{"role": "user", "content": "仕事の熱意を取り戻すためのアイデアを5つ挙げてください。"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)

with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=16384,  # thinking model: leave room for the reasoning before the answer
        do_sample=True,
        temperature=0.6,
        top_p=0.95,
    )[0][inputs["input_ids"].shape[-1]:]

for m in tokenizer.parse_harmony_message(output_ids.tolist()):
    print(f"[{tokenizer.decode(m.channel.token_ids)}]", tokenizer.decode(m.content.token_ids))

💡 Tip Recommended sampling: temperature=0.6, top_p=0.95 (default in generation_config.json).

Acknowledgements

We thank LLM-jp, led by the Research and Development Center for Large Language Models at the National Institute of Informatics, for developing the llm-jp-4 models from scratch in Japan and releasing them under a generous open license. This model is built on llm-jp/llm-jp-4-32b-a3b-base, and part of our SFT data comes from llm-jp/llm-jp-4-thinking-sft-data.

How to Cite

@misc{elyza2026thinking,
    title={elyza/ELYZA-Thinking-1.0-llm-jp-4-32b-a3b},
    url={https://huggingface.co/elyza/ELYZA-Thinking-1.0-llm-jp-4-32b-a3b},
    author={Xin Zhao, Congda Ma, Satoshi Tohda, Tasavat Trisitichoke, Akira Sasaki, Masato Hirakawa, Kota Kakiuchi},
    year={2026},
}

Citations

@misc{llm-jp-4-32b-a3b-base,
    title = {llm-jp/llm-jp-4-32b-a3b-base},
    url = {https://huggingface.co/llm-jp/llm-jp-4-32b-a3b-base},
    author = {Research and Development Center for Large Language Models},
    month = {April},
    year = {2026}
}