This repository contains the Multimodal Large Language Model (LLM) component of Step-Audio. It is a 130 billion parameter multimodal LLM that is responsible for understanding and generating human speech. The model is specifically designed to seamlessly integrate functions such as speech recognition, semantic understanding, dialogue management, voice cloning, and speech generation.
| Model | Factuality (% ↑) | Relevance (% ↑) | Chat Score ↑ |
|---|---|---|---|
| GLM4-Voice | 54.7 | 66.4 | 3.49 |
| Qwen2-Audio | 22.6 | 26.3 | 2.27 |
| Moshi* | 1.0 | 0 | 1.49 |
| Step-Audio-Chat | 66.4 | 75.2 | 4.11 |
*Note: Moshi are marked with "*" and should be considered for reference only.
| Model | Llama Question | Web Questions | TriviaQA* | ComplexBench | HSK-6 |
|---|---|---|---|---|---|
| GLM4-Voice | 64.7 | 32.2 | 39.1 | 66.0 | 74.0 |
| Moshi | 62.3 | 26.6 | 22.8 | - | - |
| Freeze-Omni | 72.0 | 44.7 | 53.9 | - | - |
| LUCY | 59.7 | 29.3 | 27.0 | - | - |
| MinMo | 78.9 | 55.0 | 48.3 | - | - |
| Qwen2-Audio | 52.0 | 27.0 | 37.3 | 54.0 | - |
| Step-Audio-Chat | 81.0 | 75.1 | 58.0 | 74.0 | 86.0 |
Note: Results marked with "*" on TriviaQA dataset are considered for reference only.
TriviaQA dataset marked with "*" indicates results are for reference only.
| Category | Instruction Following | Audio Quality | ||
|---|---|---|---|---|
| GLM-4-Voice | Step-Audio | GLM-4-Voice | Step-Audio | |
| Languages | 1.9 | 3.8 | 2.9 | 3.3 |
| Role-playing | 3.8 | 4.2 | 3.2 | 3.6 |
| Singing / RAP | 2.1 | 2.4 | 2.4 | 4 |
| Voice Control | 3.6 | 4.4 | 3.3 | 4.1 |
For more information, please refer to our repository: Step-Audio.
model-00006.safetensors 10.0 GBmodel-00009.safetensors 10.0 GBmodel-00012.safetensors 10.0 GBmodel-00015.safetensors 10.0 GBmodel-00018.safetensors 10.0 GBmodel-00021.safetensors 10.0 GBmodel-00024.safetensors 10.0 GBmodel-00003.safetensors 10.0 GBmodel-00005.safetensors 9.9 GBmodel-00008.safetensors 9.9 GBmodel-00011.safetensors 9.9 GBmodel-00014.safetensors 9.9 GBmodel-00017.safetensors 9.9 GBmodel-00020.safetensors 9.9 GBmodel-00023.safetensors 9.9 GBmodel-00026.safetensors 9.9 GBmodel-00002.safetensors 9.9 GBmodel-00001.safetensors 9.9 GBmodel-00007.safetensors 9.6 GBmodel-00010.safetensors 9.6 GBmodel-00013.safetensors 9.6 GBmodel-00016.safetensors 9.6 GBmodel-00019.safetensors 9.6 GBmodel-00022.safetensors 9.6 GBmodel-00025.safetensors 9.6 GBmodel-00004.safetensors 9.6 GBmodel-00027.safetensors 7.4 GBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo stepfun-ai/Step-Audio-Chat ./model-folder