Haas-ja-v1.0 GGUF
Quantized GGUF conversions of third-intelligence/Haas-ja-v1.0, a Japanese speech recognition model, for Haas-ASR-Java.
[!NOTE] This is an unofficial conversion, not affiliated with or endorsed by Third Intelligence. llama.cpp does not know this model's architecture and cannot run these files.
Files
Both files use Q4_K, ggml's 4-bit format, for the large matrices of the encoder. Neither uses Q6_K.
| File | Stays in bfloat16 | Size | Java heap |
|---|---|---|---|
Haas-ja-v1.0-Q4_K_S.gguf |
Attention, decoder, joint network, norms, biases | 0.54 GB | 1.5 GB |
Haas-ja-v1.0-Q4_K_XS.gguf |
Decoder, joint network, norms, biases | 0.36 GB | 0.8 GB |
The original weights take 1.2 GB. Q4_K_S is the safer choice, since it changes the output less. The heap is the smallest that worked on about 20 s of audio.
These names are specific to this conversion. Q4_K_XS is not llama.cpp's IQ4_XS, which is a different format, and both files record their file type as Q4_K_S. The tensor names and the metadata are also this project's own, so other GGUF runtimes for this model may not read these files.
Usage
You need JDK 25 or later. No Python or GPU is needed.
git clone https://github.com/kishida/Haas-ASR-Java
cd Haas-ASR-Java
curl -L -O https://huggingface.co/kishida/Haas-ja-v1.0-GGUF/resolve/main/Haas-ja-v1.0-Q4_K_S.gguf
# Transcribe a WAV file
java --add-modules jdk.incubator.vector src/main/java/haas/Main.java Haas-ja-v1.0-Q4_K_S.gguf example.wav
# Or open the UI, and drop the model and an audio file on the window
java --add-modules jdk.incubator.vector src/main/java/haas/Main.java
On a 16-thread desktop CPU it transcribes about 4 times faster than real time.
Accuracy
The accuracy of these files has not been measured on a benchmark. They were compared with the original model, running in PyTorch, on four clips of 8 to 124 s, mostly synthesized speech:
Q4_K_S: the transcripts were identical.Q4_K_XS: one clip differed by one character (我輩 and 吾輩). The others were identical.
For the original model's results, see its model card.
How they were made
They were converted with tools/export_gguf.py, which uses the method of ggml's quantize_row_q4_K_ref:
python tools/export_gguf.py third-intelligence/Haas-ja-v1.0 Haas-ja-v1.0-Q4_K_S.gguf
python tools/export_gguf.py third-intelligence/Haas-ja-v1.0 Haas-ja-v1.0-Q4_K_XS.gguf --quantize-attention
License
Apache License 2.0, the same as the original model.