Index-Nailong-2B-GGUF
Official GGUF conversion of IndexTeam/Index-Nailong-2B, part of the Index-Translate multilingual translation model family (150 languages, terminology/format-constrained translation, controlled dubbing translation, long-document translation).
- Technical report: Index-Translate: A Multilingual Translation Model Family
- Code: github.com/bilibili/Index-Translate
Converted with llama.cpp (master, 2026-10); static post-training quantization, all bit-widths in this single repository.
Provided files
(sorted by size)
| File | Size | Notes |
|---|---|---|
| mmproj-Q8_0 | 0.36 GB | multi-modal projector supplement |
| mmproj-f16 | 0.67 GB | multi-modal projector supplement |
| Q2_K | 0.97 GB | 2-bit, significant quality loss |
| Q3_K_S | 1.02 GB | 3-bit, noticeable quality loss |
| Q3_K_M | 1.10 GB | 3-bit, noticeable quality loss |
| Q3_K_L | 1.16 GB | 3-bit, noticeable quality loss |
| IQ4_XS | 1.20 GB | 4-bit I-quant, small |
| Q4_K_S | 1.21 GB | 4-bit, slightly smaller/faster |
| Q4_K_M | 1.27 GB | 4-bit, good balance, recommended |
| Q5_K_S | 1.37 GB | 5-bit, low quality loss |
| Q5_K_M | 1.41 GB | 5-bit, low quality loss |
| Q6_K | 1.56 GB | 6-bit, very low quality loss |
| Q8_0 | 2.01 GB | 8-bit, near-lossless |
| f16 | 3.78 GB | 16-bit, lossless conversion baseline |
The mmproj files are the multi-modal projector for the vision tower; they are only needed for image-input use with llama-mtmd-cli. For text-only translation, the plain GGUF files suffice.
Usage
# Run the recommended Q4_K_M directly:
llama serve -hf IndexTeam/Index-Nailong-2B-GGUF:Q4_K_M
llama cli -hf IndexTeam/Index-Nailong-2B-GGUF:Q4_K_M
Translation prompt format (greedy decoding, temperature=0 recommended; use the chat template with enable_thinking: false):
请将以下文本翻译为{{target-language}}，直接输出翻译结果，不要进行任何解释。
{{source-text}}
# llama.cpp: run the recommended Q4_K_M with the chat template and thinking disabled
llama-cli -hf IndexTeam/Index-Nailong-2B-GGUF:Q4_K_M --jinja --chat-template-kwargs '{"enable_thinking":false}' --temp 0
Prompting & constrained translation (instTrans)
Beyond plain translation, the models follow the instTrans constrained-translation format. The official client wraps requests into the canonical structure 【源文】<text> + numbered 1. 【硬性要求】<hard constraints> + 2. 【注意】<soft constraints> + suffix instructions:
- Hard constraints (binary, must hold): strict terminology glossary enforcement (e.g.
碳纤维:carbon fiber, 抗裂缝:crack resistance), and format/structure preservation for JSON/CSV/code/placeholders. - Soft constraints (graded): tone & style adaptation (e.g. formal business-email register), domain/word-sense disambiguation (e.g. plant -> 工厂 in an industrial context), cross-sentence consistency, LaTeX preservation.
- Syllable-controlled translation (dubbing): the Index-Homura checkpoints (IndexTeam/Index-Homura-2B, IndexTeam/Index-Homura-9B) strictly respect a target syllable budget and can be combined with glossaries.
Full prompt reference: github.com/bilibili/Index-Translate (Instruction Following section, docs/prompts.md, inference/llm/cases/).
See the base model card for the full instTrans constrained-translation format and serving presets.
Consistency validation
Before release, each quantization tier was validated on GPU (NVIDIA A100) against the F16 conversion: per-token KL divergence / RMS delta-p via llama-perplexity, plus greedy-generation spot checks against the original BF16 weights (transformers reference). Generation outputs of Q4_K_M matched the reference almost verbatim.
Converted and published by the Index team, 2026-10-03.