Fun-CosyVoice3-0.5B-2512-GGUF

Unofficial community GGUF release for CosyVoice3-2512.

This model pack is intended for use with cosyvoice.cpp.

Important Notice

• This is not an official CosyVoice release.
• The implementation is maintained by an independent community developer.
• The C++ runtime project (cosyvoice.cpp) is MIT.
• Model artifacts in this pack remain Apache-2.0.

Available Files

Naming Changes

The following quantizations have been renamed:

• Q6_K_S → Q6_K
• Q5_K_S → Q5_K_XXS
• Q4_K_S → Q4_K_XXS
• Q3_K_S → Q3_K_XXS
• Q2_K_S → removed (mostly noisy and unusable)

Current Variants

Quick Recommendations

K-quant quality relationship within each series (high to low): K_M > K_S > K_XXS

• Default recommendation: Q8_0 (near-lossless listening quality in current tests, with much smaller size than F16)
• High-quality alternatives: F16 and Q6_K
• Usable mid-tier options: Q5_1, Q5_K_M, Q5_K_S, Q5_0, Q5_K_XXS
• Quality drops but still acceptable: Q4_K_M, Q4_K_S
• Audible artifacts: Q4_1, Q4_K_XXS, Q4_0
• Further quality degradation: Q3_K_M, Q3_K_S, Q3_K_XXS
• Extreme compression only: Q2_K_L, Q2_K

Subjective listening notes from current tests:

• Q8_0: quality remains strong with little audible loss in typical samples.
• Q6_K: still sounds good and is often a practical choice.
• Q5_K_M / Q5_K_S (new): maintain decent quality, good results.
• Q5_K_XXS (formerly Q5_K_S): slightly higher than but close to Q4_K_M.
• Q5 family (Q5_1 / Q5_K_M / Q5_K_S / Q5_0 / Q5_K_XXS): generally usable with moderate degradation.
• Q4_K_M / Q4_K_S (new): quality drops compared to Q5, occasional muffled audio, but mostly acceptable.
• Q4 family (including Q4_1 / Q4_K_XXS / Q4_0): audible pronunciation artifacts, often muffled.
• Q3_K_M / Q3_K_S (new): further quality degradation, audio is muffled, slightly below but close to Q4_K_XXS.
• Q3_K_XXS (formerly Q3_K_S): aggressive quantization, limited quality.
• Q2_K_L (new): significant quality drop, quieter and muffled audio with noise.
• Q2_K (new): severe artifacts and noise, but speech content can still be discerned.
• Q2_K_S (removed): mostly degrades into noise and is usually unusable.

Runtime Requirements

• Built for cosyvoice.cpp GGUF inference pipeline.
• Parts of the runtime currently require AVX2 support.
• GPU backend behavior depends on GGML backend and driver stack.

Basic Usage

cosyvoice-cli \
  --model CosyVoice3-2512_Q8_0.gguf \
  --prompt-speech prompt_speech.gguf \
  --text "Hello from CosyVoice" \
  --output out.wav

--prompt-speech expects a prompt-speech file in GGUF format (for example prompt_speech.gguf).

For runtime usage and documentation, see the cosyvoice.cpp repository:

• https://github.com/Lourdle/cosyvoice.cpp

Provenance

• This work follows the architecture and inference behavior of upstream CosyVoice releases, then re-implements the pipeline for C++/GGML deployment.
• Tokenizer implementation is adapted from llama.cpp and modified for this project.

License

• Model files in this repository: Apache-2.0
• Runtime code (cosyvoice.cpp): MIT