Index-Echo-S2ST-2B-GGUF
Official GGUF conversion of the text LLM backbone (stlm_llm/) of IndexTeam/Index-Echo-S2ST-2B (v4.1), part of the Index-Echo speech-to-speech translation (S2ST) model family by bilibili.
The v4.1 package updates the multilingual translator and extends the release to twelve directions: Chinese, Japanese, or English speech input, dubbed into English, Spanish, Japanese, Korean, or Chinese.
Note: LLM backbone only. The audio tower, connector and speech-synthesis stack are not included - llama.cpp cannot run the full speech pipeline. For full speech-to-speech translation (S2ST) inference, use the original checkpoint or the FP8 build in Index-Echo-S2ST-2B-FP8, which ships the complete pipeline with an FP8-quantized LLM.
Converted with llama.cpp (master, 2026-10-13); static post-training quantization, all bit-widths in this single repository.
Provided files
(sorted by size)
| File | Size | Notes |
|---|---|---|
| mmproj-Q8_0 | 0.36 GB | multi-modal projector supplement |
| mmproj-f16 | 0.67 GB | multi-modal projector supplement |
| Q2_K | 0.97 GB | 2-bit, significant quality loss |
| Q3_K_S | 1.02 GB | 3-bit, noticeable quality loss |
| Q3_K_M | 1.10 GB | 3-bit, noticeable quality loss |
| Q3_K_L | 1.16 GB | 3-bit, noticeable quality loss |
| IQ4_XS | 1.20 GB | 4-bit I-quant, small |
| Q4_K_S | 1.21 GB | 4-bit, slightly smaller/faster |
| Q4_K_M | 1.27 GB | 4-bit, good balance, recommended |
| Q5_K_S | 1.37 GB | 5-bit, low quality loss |
| Q5_K_M | 1.41 GB | 5-bit, low quality loss |
| Q6_K | 1.56 GB | 6-bit, very low quality loss |
| Q8_0 | 2.01 GB | 8-bit, near-lossless |
| f16 | 3.78 GB | 16-bit, lossless conversion baseline |
The mmproj files are the multi-modal projector for the vision tower; they are only needed for image-input use with llama-mtmd-cli. For text-only translation, the plain GGUF files suffice. (The v4.1 package no longer bundles this tower inside stlm_llm/; the mmproj files are carried over unchanged from the initial release.)
Usage
# Run the recommended Q4_K_M directly:
llama serve -hf IndexTeam/Index-Echo-S2ST-2B-GGUF:Q4_K_M
llama cli -hf IndexTeam/Index-Echo-S2ST-2B-GGUF:Q4_K_M
Consistency validation
Before release, each quantization tier was validated against the F16 conversion: per-token KL divergence / RMS delta-p via llama-perplexity (CPU build of llama.cpp), plus greedy-generation spot checks against the original BF16 weights (transformers reference, CPU).
| Quant | PPL (fixed corpus) | Mean KLD | RMS Δp | Same top-1 |
|---|---|---|---|---|
| IQ4_XS | 4.9447 | 0.071674 | 7.86% | 91.13% |
| Q2_K | 10.6969 | 1.090653 | 33.49% | 63.01% |
| Q3_K_L | 5.4337 | 0.217771 | 13.94% | 84.47% |
| Q3_K_M | 5.4265 | 0.244748 | 14.17% | 83.30% |
| Q3_K_S | 6.0653 | 0.349583 | 17.79% | 80.43% |
| Q4_K_M | 4.9306 | 0.070487 | 7.82% | 90.93% |
| Q4_K_S | 4.8496 | 0.074472 | 7.77% | 90.74% |
| Q5_K_M | 4.7035 | 0.020143 | 4.31% | 94.98% |
| Q5_K_S | 4.6823 | 0.021831 | 4.37% | 94.85% |
| Q6_K | 4.7519 | 0.008402 | 2.67% | 96.48% |
| Q8_0 | 4.7325 | 0.001318 | 1.20% | 99.02% |
| f16 | 4.7059 | (reference) | - | - |
Converted and published by the Index team, 2026-10-13.