LFM3-Audio-2B-GGUF
GGUF version of LFM3-Audio-2B for local inference on CPUs and GPUs.
LFM3-Audio-2B is a compact audio model that runs locally on laptops and edge devices. It takes text or audio as input and responds with text, speech, or both.
Find architecture and usage details in the original model card.
How to run LFM3-Audio-2B
Recommended sampling: greedy decoding (temperature 0) for text output. For speech output, audio temperature 1.0, audio top_k 128 and audio repetition penalty 1.1.
- Llama.app (Use this model > Llama in the menu), for macOS and Windows: audio input. Requires llama.cpp version v0.8.0 or newer.
- Ollama (
ollama run lfm3-audio), for macOS: audio input. Requires Ollama version v0.40.12 or newer.
Use with llama.cpp
[!NOTE] Supported in llama.cpp b11960 (v0.8.0) and newer, including real-time voice conversations (with tool calls) on the
/v1/realtimeWebSocket API ofllama serve.
Add a quantization tag to pick a file. With -hf, llama.cpp also downloads the audio projector.
Start a web chat UI and an OpenAI-compatible API at http://localhost:8080:
llama serve -hf LiquidAI/LFM3-Audio-2B-GGUF:Q4_K_M --temp 0 --port 8080
To ask one question from the command line, use llama cli.
Speech recognition: use the system prompt Perform ASR in <language>., or Perform ASR. to let the model infer the language.
llama cli -hf LiquidAI/LFM3-Audio-2B-GGUF:Q4_K_M \
--audio speech.wav -sys "Perform ASR in English." -p "<__media__>" --temp 0
Speech synthesis: --tts-lang takes a language code (default en) and --tts-voice is female (default) or male. The audio sampling parameters of the original model card are stored in the GGUF.
llama-tts -m LFM3-Audio-2B-Q4_K_M.gguf -mm mmproj-LFM3-Audio-2B-Q8_0.gguf \
-p "Hello world" --tts-voice male -o out.wav
More options in the llama.cpp guide.