Index-Echo-S2ST-9B-FP8-v4rft

Official FP8 (W8A8) quantization of IndexTeam/Index-Echo-S2ST-9B (v4rft), part of the Index-Echo speech-to-speech translation (S2ST) model family by bilibili.

The v4rft package updates the multilingual translator and extends the release to twelve directions: Chinese, Japanese, or English speech input, dubbed into English, Spanish, Japanese, Korean, or Chinese.

This repository mirrors the original checkpoint layout; only the text LLM backbone (stlm_llm/) is quantized to FP8 - the audio tower, connector, and all speech-synthesis components remain in BF16. Use it exactly like the original repo (same modeling_dubbing.py / configs).

Quantization

• Scheme: FP8_DYNAMIC (FP8 E4M3 weights, per-token dynamic FP8 activations), produced with llm-compressor (quantization_scheme recorded in recipe.yaml).
• All Linear layers of the language model backbone (stlm_llm/) are quantized; the audio tower, connector, lm_head, embeddings and all other pipeline components are kept in BF16.
• Format: compressed-tensors safetensors - load directly with vLLM (quantization="compressed-tensors") or transformers.

Consistency validation

Measured on an NVIDIA A100 against the original BF16 checkpoint (greedy decoding, fixed bilingual corpus):

Usage

Identical to the original checkpoint - clone this repo and follow the README / modeling_dubbing.py of the base model (IndexTeam/Index-Echo-S2ST-9B). The quantized LLM backbone loads via compressed-tensors; make sure compressed-tensors (or vLLM / a recent transformers) is installed.

Quantized and published by the Index team, 2026-10-15.