Index-Translate-35B-A3B-preview-APEX-GGUF

APEX GGUF quantizations of IndexTeam/Index-Translate-35B-A3B-preview, quantized by alphaZimuth.

Index-Translate-35B-A3B-preview is a 35B-total / 3B-active Mixture-of-Experts (MoE) multilingual translation model from the Index-Translate family, built on Qwen3.5.

The model supports translation across approximately 150 languages and is designed to follow translation instructions involving terminology, formatting, style, structure, context, and output-length constraints.

This repository provides four APEX-I quantization tiers using importance-matrix-assisted APEX quantization.

Upstream

• Base model: IndexTeam/Index-Translate-35B-A3B-preview
• Official GGUF: IndexTeam/Index-Translate-35B-A3B-preview-GGUF
• Code: bilibili/Index-Translate
• Technical report: Index-Translate: A Multilingual Translation Model Family
• arXiv: 2609.40181

This is a third-party APEX quantization release and is not an official IndexTeam repository.

---

Available Quantizations

Four APEX-I quantization tiers are currently available:

For most users, APEX-I-Compact is the recommended starting point.

---

Quantization Details

This repository uses APEX-I importance-matrix-assisted quantization with llama.cpp.

The importance matrix used for these quantizations was generated specifically for Index-Translate-35B-A3B-preview, using multilingual calibration data designed to reflect translation workloads rather than generic text completion.

Two complementary calibration corpora were used:

• a multilingual corpus designed for broad and relatively uniform language coverage
• a weighted corpus placing greater emphasis on 22 core languages

Both calibration sets contain 100% unique documents and together cover 146 of the model's 150 supported languages.

The calibration samples primarily follow a translation-oriented structure:

Translation instruction
Source text
Reference translation

Additional constrained-translation samples were included for tasks such as terminology consistency, JSON/placeholder preservation, and structured output.

The two calibration runs produced:

Uniform multilingual imatrix : 229 chunks
Core-language weighted imatrix: 220 chunks
                              ──────────
Final merged imatrix          : 449 chunks

The two importance matrices were merged by combining their accumulated activation statistics, producing a single calibration matrix representing both broad multilingual coverage and higher-density core-language data.

The resulting merged imatrix was then used for all four APEX-I quantization tiers:

IndexTeam/Index-Translate-35B-A3B-preview
                     ↓
              GGUF Conversion
                     ↓
        Multilingual Imatrix Calibration
                 ↙           ↘
       Uniform Coverage    Core22 Weighted
            229 chunks       220 chunks
                 ↘           ↙
            Merged Imatrix
              449 chunks
                     ↓
             APEX-I Quantization
                     ↓
      Nano / Mini / Compact / Quality

Note

Perplexity reported during imatrix generation is used only as a calibration health diagnostic and is not treated as a model-quality benchmark. The final quantized models should be evaluated independently.

---

Usage

A recent version of llama.cpp is recommended.

For text translation, no multimodal projector is required.

Example using APEX-I-Compact:

llama-cli \
    -m Index-Translate-35B-A3B-preview-APEX-I-Compact.gguf \
    --jinja \
    --chat-template-kwargs '{"enable_thinking":false}' \
    --temp 0

Or start an OpenAI-compatible server:

llama-server \
    -m Index-Translate-35B-A3B-preview-APEX-I-Compact.gguf \
    --jinja \
    --chat-template-kwargs '{"enable_thinking":false}' \
    --temp 0

For translation, greedy decoding / temperature=0 is recommended.

Thinking should normally be disabled:

enable_thinking = false

If you require image-input functionality, refer to the official IndexTeam GGUF repository for the corresponding multimodal projector files.

---

Translation Prompt

For standard translation, the recommended prompt format is:

请将以下文本翻译为{target-language}，直接输出翻译结果，不要进行任何解释。

{source-text}

Example:

请将以下文本翻译为英语，直接输出翻译结果，不要进行任何解释。

你好，世界。今天天气不错。

---

Prompting & Constrained Translation

Beyond standard translation, Index-Translate supports instTrans constrained translation.

The model can follow both hard and soft translation constraints.

Hard constraints

Hard constraints are requirements that should be strictly preserved or followed, such as:

• terminology / glossary enforcement
• JSON, CSV, Markdown, and code structure preservation
• variable and placeholder preservation
• required terminology mappings

Soft constraints

Soft constraints can be used to guide:

• tone and writing style
• domain-specific wording
• contextual disambiguation
• cross-sentence consistency

Hard and soft constraints can be combined in the same translation request.

For the complete prompt specification and additional examples, refer to the official Index-Translate repository.

---

About the Model

Index-Translate-35B-A3B-preview is part of the Index-Translate multilingual translation model family.

Its main capabilities include:

• multilingual text translation
• terminology-constrained translation
• structured-format preservation
• style-controlled translation
• context-aware translation
• multilingual instruction following

For tasks that require strict target syllable counts, use the dedicated Index-Homura models.

For native full-document translation workflows, refer to the dedicated Index-NativeLong models.

---

Upstream Evaluation

Model-quality and translation benchmarks are provided by the IndexTeam in the original model card and technical report.

Those results apply to the original Index-Translate-35B-A3B-preview model and should not be interpreted as independent benchmark results for these APEX GGUF quantizations.

Quantization can introduce differences in output quality and model behavior, especially at more aggressive compression levels.

In particular, APEX-I-Nano should be considered a near-limit quantization intended primarily for environments where memory usage is the main constraint.

---

Credits

Special thanks to:

• IndexTeam / Index LLM Team for developing and releasing Index-Translate.
• Bilibili for the Index-Translate project and inference tools.
• LocalAI for the APEX quantization framework.
• llama.cpp contributors for GGUF conversion, quantization, and inference support.
• The wider open-source LLM community.

APEX GGUF quantization by alphaZimuth.

---

Citation

If you use the original Index-Translate model in research, please refer to the official technical report:

@techreport{indextranslate2026,
  author={Tianjiao Li and Mengran Yu and Chenyu Shi and Lusheng Zhang and
          Qisi Chen and Yanshan Zhou and Ji Qi and Jingying Liu and
          Yuang Feng and Ziang Cui and Tianxing Yan},
  title={Index-Translate: A Multilingual Translation Model Family --- Text, Speech, Controlled Dubbing, and Long-Document Translation},
  institution={Index LLM Team},
  year={2026},
  month={September},
  url={https://arxiv.org/abs/2609.40181}
}

---

License

Apache-2.0

Please also review the license and usage terms of the original IndexTeam/Index-Translate-35B-A3B-preview model.

---

Related Models

Original model

IndexTeam/Index-Translate-35B-A3B-preview

Official GGUF release

IndexTeam/Index-Translate-35B-A3B-preview-GGUF

APEX GGUF quantization

alphaZimuth/Index-Translate-35B-A3B-preview-APEX-GGUF