Try LFM • Docs • Discord

LFM2.5-24B-A2B-GGUF

LFM2 is a new generation of hybrid models developed by Liquid AI, specifically designed for edge AI and on-device deployment. It sets a new standard in terms of quality, speed, and memory efficiency.

Find more details in the original model card: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B

🏃 How to run LFM2

Example usage with llama.cpp:

llama-cli -hf LiquidAI/LFM2.5-24B-A2B-GGUF \
    --temp 0.1 --top-k 50 --repeat-penalty 1.05

The sampling parameters are the ones recommended for the base model (its generation_config.json); each GGUF also carries them as general.sampling.* metadata, so llama.cpp applies them by default.

QAD Q4_0 GGUF

The Q4_0 build in this repository is the Quantization-Aware Distillation (QAD) checkpoint, LFM2.5-24B-A2B-QAD-Q4_0.gguf. It was trained to recover the accuracy lost to Q4_0 quantization.

Example usage with llama.cpp:

llama-cli -hf LiquidAI/LFM2.5-24B-A2B-GGUF \
    --hf-file LFM2.5-24B-A2B-QAD-Q4_0.gguf \
    --temp 0.1 --top-k 50 --repeat-penalty 1.05