Subscribe to my newsletter · Support

LFM3-1.7B-GSQ-GGUF

Three GGUF checkpoints of LiquidAI/LFM3-1.7B, compressed using SLQ + GSQ or RCO + GSQ.

Models

All three use custom mixed precision. Filename quantization suffixes identify the main format family; the heatmaps below show the actual tensor formats. Select the exact filename when downloading.

Precision by layer

GSQ-Q6_K_RCO

GSQ-Q6_K_SLQ

GSQ-1GB-IQ4_XS_RCO

Held-out evaluation

EAR means Expected Acceptance Rate, measured against the BF16 reference. Each language uses 64 sequences of 2,048 tokens. English uses the existing FineWeb-Edu held-out set; French, Japanese, and Hindi use FineWeb2 test text. Every checkpoint in a language chart uses the same text and reference.

EAR measured on different text is not directly comparable. Compare checkpoints within each language chart.

English

French

Japanese

Hindi

Run with llama.cpp

Use a llama.cpp build with LFM3 support. For example, download the RCO + GSQ Q6_K-family checkpoint:

hf download kaitchup/LFM3-1.7B-GSQ-GGUF \
  LFM3-1.7B-GSQ-Q6_K_RCO.gguf --local-dir .

llama-server \
  --model LFM3-1.7B-GSQ-Q6_K_RCO.gguf \
  --n-gpu-layers 99 --ctx-size 8192 --port 8080

Use another exact filename from the table to select a different checkpoint.

Papers and credits

Methods: SLQ · RCO · GSQ.

Credit goes to Liquid AI for the base model and official GGUFs, AtomicChat for its GGUFs, and the authors of SLQ, RCO, and GSQ.

License: LFM Open License v1.0. See NOTICE for attribution and modifications.

If these models are useful, support the work or subscribe to The Kaitchup.