MOSS-Transcribe-Diarize GGUF

GGUF files of OpenMOSS-Team/MOSS-Transcribe-Diarize for llama.cpp.

Upstream llama.cpp does not support this model yet. Use the fork github.com/mrom1/llama.cpp-MOSS-TD (branch moss-td; its releases have the usual prebuilt binaries). With it, llama-server and llama-mtmd-cli run the model as they are.

MOSS-Transcribe-Diarize (0.9B, Apache-2.0) writes the transcript of a recording with speaker labels and timestamps, up to 90 minutes in one pass:

[0.48][S01]Welcome everyone[1.66][12.26][S02]The new transcription pipeline is ready[13.81]

Files

SHA256SUMS lists the checksums.

Usage

With the fork's binaries:

llama-server -hf mrom1/MOSS-Transcribe-Diarize-GGUF:Q8_0 -c 16384 -ctk q8_0 -ctv q8_0 -fa on -ub 512

Send a chat message with the audio and the instruction the model was trained on, temperature 0:

请将音频转写为文本，每一段需以起始时间戳和说话人编号（[S01]、[S02]、[S03]…）开头，正文为对应的语音内容，并在段末标注结束时间戳，以清晰标明该段语音范围。

• -c must hold the audio (12.5 tokens per second), the time anchors and the transcript: about seconds * 12.5 * 1.6 + 4096. Five minutes: ~10k; one hour: ~70k (about 6 GB of GPU memory with -ctk q8_0 -ctv q8_0; a 4-bit cache breaks the output).
• llama-server takes at most 100 MB per request: send long recordings as FLAC or MP3, or in pieces. llama-mtmd-cli --audio file reads from disk.

More: docs/multimodal/moss-transcribe-diarize.md in the fork.

Converting yourself

With the fork's convert_hf_to_gguf.py:

python convert_hf_to_gguf.py <model dir> --outfile MOSS-Transcribe-Diarize-F16.gguf --outtype f16
python convert_hf_to_gguf.py <model dir> --mmproj --outfile mmproj-MOSS-Transcribe-Diarize-Q8_0.gguf --outtype q8_0
llama-quantize MOSS-Transcribe-Diarize-F16.gguf MOSS-Transcribe-Diarize-Q8_0.gguf Q8_0

The original config.json sets adaptor_input_dim, which the model's own config class declares read-only, so transformers cannot load the tokenizer; convert from a copy of the model directory whose config.json leaves that key out.

License

Apache-2.0, as the original model by OpenMOSS-Team. These files are a format conversion of its weights.