RakutenAI-3.0

Model Description

RakutenAI-3.0 is an approximately 700 billion parameter Mixture of Experts (MoE) model optimized for Japanese. Developed by leveraging the best from the open-source community and building on Rakuten’s high-quality, bilingual original data, engineering and research, it offers a superior grasp of Japanese language and culture.

For more details, please check the announcement at:

• ENGLISH Version
• JAPANESE Version

Model Details

• Developed by: Rakuten Group, Inc.
• Language(s): Japanese, English
• License: Apache License, Version 2.0
• Model Architecture: Mixture of Experts (MoE)
• Total Parameters: 671B
• Activated Parameters per Token: 37B
• Context Length: 128K

How to Run Locally

Inference with SGLang

• Recommended DOCKER Image: lmsysorg/sglang:v0.5.6.post2

python -m sglang.launch_server \
    --model-path Rakuten/RakutenAI-3.0 \
    --tp 8 \
    --mem-fraction-static 0.85 \
    --trust-remote-code \
    --show-time-cost

Limitations and Bias

RakutenAI-3.0 can generate human-like text across a wide range of topics. However, like all large language models, it has limitations and may produce biased, inaccurate, unsafe outputs, or outputs that may infringe the rights of third parties. Please exercise caution and judgment when interacting with this model, and ensure appropriate safeguards are in place for production deployments.

Citation

@misc{rakutengroup2026rakutenai3.0,
  author = {Rakuten Group, Inc.},
  title = {RakutenAI-3.0},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Rakuten},
}

Release Notes

Added on November 5, 2026

RakutenAI-3.0 was developed in two phases. First, DeepSeek-V3-0324, an open-weight model checkpoint, underwent continual pre-training (CPT) on a diverse English-Japanese corpus using publicly available and synthetic data. Model merging was used to limit the loss of existing model capabilities after continual pre-training. Next, we performed multi-staged supervised fine-tuning (SFT) covering instruction following, debiasing, multi-turn dialogue and alignment. The SFT stages were interleaved with on-policy data generation and rejection sampling to further reduce catastrophic forgetting and enhance model capabilities.

Academic Japanese-Language Benchmarks

Metric comparison between DeepSeek-V3-0324 and RakutenAI-3.0 on Japanese-language benchmarks. Evaluation was performed using the swallow-evaluation-instruct framework. RakutenAI-3.0 scores higher on Japanese cultural questions (JamC-QA) and instruction following (M-IFEval), while DeepSeek-V3-0324 remains stronger on graduate-level reasoning (MMLU-ProX) and competition-level math (MATH-100) - a trade-off between improved Japanese-focused performance and some general reasoning and math capabilities of the base checkpoint.

Internal Safety Evaluation

Before training, we thoroughly evaluated the DeepSeek base model for security risks in an isolated, sandboxed environment and did not identify any security issues. To ensure model output safety, we created extra data to substantially reduce bias and used it to improve the model. We also built a separate test dataset to measure the effectiveness of this effort.

The neutrality score represents the magnitude of bias, so a lower neutrality score means the model presents information with more impartiality, whereas a higher score means it tends to lean toward a particular viewpoint. Deflection is the share of outputs that evade the user's request - for example, by refusing without justification or changing subject. Language coherence rewards the outputs that are fluent, grammatically well-formed, and on topic in the requested language. Lower scores are better for deflection; higher is better for language coherence. This evaluation uses single-turn prompts. Compared with DeepSeek-V3-0324, RakutenAI-3.0 reduces the neutrality score by 78.8% and the deflection score by 83.4%, while improving language coherence. gpt-4o is included as an external reference and has slightly lower neutrality and deflection scores in this evaluation.

Disclaimer regarding the European Union (EU) Market

This model is NOT intended for, and is NOT made available to, users or entities residing within the European Union (EU) or the European Economic Area (EEA). The Provider has not placed, and does not intend to place, this model on the EU market.

Contact

For questions or feedback, please open an issue on this repository or visit ai.rakuten.com.