Index-Echo-S2ST-9B-GGUF
Official GGUF conversion of the text LLM backbone (stlm_llm/) of IndexTeam/Index-Echo-S2ST-9B (v4rft), part of the Index-Echo speech-to-speech translation (S2ST) model family by bilibili.
The v4rft package updates the multilingual translator and covers twelve directions: Chinese, Japanese, or English speech input, dubbed into English, Spanish, Japanese, Korean, or Chinese.
Note: LLM backbone only. The audio tower, connector and speech-synthesis stack are not included - llama.cpp cannot run the full speech pipeline. For full speech-to-speech translation (S2ST) inference, use the original checkpoint or the FP8 build in Index-Echo-S2ST-9B-FP8, which ships the complete pipeline with an FP8-quantized LLM.
Converted with llama.cpp (v0.26.0, 2026-10-15); static post-training quantization, all bit-widths in this single repository.
Provided files
(sorted by size)
| File | Size | Notes |
|---|---|---|
| Q2_K | 3.83 GB | 2-bit, significant quality loss |
| Q3_K_S | 4.26 GB | 3-bit, noticeable quality loss |
| Q3_K_M | 4.62 GB | 3-bit, noticeable quality loss |
| Q3_K_L | 4.93 GB | 3-bit, noticeable quality loss |
| IQ4_XS | 5.23 GB | 4-bit I-quant, small |
| Q4_K_S | 5.35 GB | 4-bit, slightly smaller/faster |
| Q4_K_M | 5.63 GB | 4-bit, good balance, recommended |
| Q5_K_S | 6.31 GB | 5-bit, low quality loss |
| Q5_K_M | 6.47 GB | 5-bit, low quality loss |
| Q6_K | 7.36 GB | 6-bit, very low quality loss |
| Q8_0 | 9.53 GB | 8-bit, near-lossless |
| f16 | 17.92 GB | 16-bit, lossless conversion baseline |
Usage
# Run the recommended Q4_K_M directly:
llama serve -hf IndexTeam/Index-Echo-S2ST-9B-GGUF:Q4_K_M
llama cli -hf IndexTeam/Index-Echo-S2ST-9B-GGUF:Q4_K_M
Consistency validation
Before release, each quantization tier was validated against the F16 conversion: per-token KL divergence / RMS delta-p via llama-perplexity (CPU build of llama.cpp), plus greedy-generation spot checks.
| Quant | PPL (fixed corpus) | Mean KLD | RMS Δp | Same top-1 |
|---|---|---|---|---|
| f16 | 3.2502 | (reference) | - | - |
| Q8_0 | 3.2507 | 0.001272 | 1.23% | 99.02% |
| Q6_K | 3.2467 | 0.012566 | 4.18% | 96.61% |
| Q5_K_M | 3.2775 | 0.016146 | 4.45% | 96.09% |
| Q5_K_S | 3.3047 | 0.025917 | 5.45% | 95.37% |
| Q4_K_M | 3.2806 | 0.035922 | 6.90% | 94.39% |
| Q4_K_S | 3.6296 | 0.090064 | 10.84% | 91.32% |
| Q3_K_L | 3.4872 | 0.092859 | 10.21% | 91.00% |
| Q3_K_M | 3.8014 | 0.151914 | 14.26% | 87.87% |
| Q3_K_S | 5.8845 | 0.535761 | 30.10% | 80.36% |
| Q2_K | 4.6526 | 0.596711 | 26.59% | 76.32% |
| IQ4_XS | 3.4390 | 0.081273 | 10.17% | 91.52% |
Converted and published by the Index team, 2026-10-16.