LFM3-1.7B, ROCmFPX GGUF
LiquidAI/LFM3-1.7B, Liquid AI's 1.7B text model for local agents, tool calling and structured output (tri-hybrid: gated short convolutions plus sliding-window and global attention, 256K context), quantized for the ROCmFPX llama.cpp fork (low-bit weight types for AMD GPUs, Vulkan and HIP/ROCm).
The recommended 6-bit file needs HarshaVardhan3002/ROCmFPX. Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds may not load it. The Q8_0 and Q4_K_M files use stock types and load in any llama.cpp build with LFM3 support. These files were checked on the CPU and have not been tested on AMD hardware yet.
Not an official release of Liquid AI. All credit for the model goes to its authors.
Files
| File | Size | Weights | KL divergence vs BF16 (lower is better) | Same top token as BF16 |
|---|---|---|---|---|
LFM3-1.7B-Q6_K_ROCMFPX.gguf (recommended) |
1.43 GB | ROCmFPX 6-bit (Q6_0_ROCMFPX with tensor scales) |
0.0132 | 94.3% |
LFM3-1.7B-Q8_0.gguf |
1.85 GB | stock llama.cpp Q8_0 | 0.0031 | 97.2% |
LFM3-1.7B-Q4_K_M.gguf |
1.10 GB | stock llama.cpp Q4_K_M | 0.0922 | 85.5% |
| stock Q6_K, for comparison (not uploaded) | 1.43 GB | stock llama.cpp Q6_K | 0.0160 | 93.8% |
- Measured on: 40 chunks of 512 tokens of WikiText-2 (test split), against the BF16 GGUF, CPU.
- The ROCmFPX 6-bit file beats stock Q6_K at the same size. Its file name says
Q6_K_ROCMFPXso that Hugging Face lists it as a 6-bit quant; the weights are the ROCmFPXQ6_0_ROCMFPXtype, not a stockQ6_K. - No ROCmFPX 4-bit file: our 4-bit types came out clearly worse than stock Q4_K_M on this model (KL divergence 1.5 to 1.7 times higher), so we ship Q4_K_M instead.
- Every file passed a short greedy answer test before upload.
How these were made
- BF16 GGUF converted with the fork's
convert_hf_to_gguf.py. - Importance matrix from the BF16 file over a general calibration text (
calibration_datav3), 512-token chunks. llama-quantize --imatrixfrom the BF16 file for every file, stock ones included. The 6-bit file was made withLLAMA_ROCMFPX_TENSOR_SCALE=1, which stores a power-of-two scale per weight tensor (the runtime applies it; no setting is needed to run the file), with--token-embedding-type q6_K.
Run
git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build -j
build/bin/llama-server -m LFM3-1.7B-Q6_K_ROCMFPX.gguf -ngl 99 --jinja -c 32768 \
--temp 0.1 --top-k 50 --repeat-penalty 1.1
Liquid AI recommends temperature 0.1, top-k 50, repetition penalty 1.1 for chat, tool use and agents, and temperature 1.0, top-p 0.95, top-k 20 for math. The model supports up to 256K tokens of context.
LFM3 support in the fork comes from llama.cpp pull request #31252. LFM3-Omni and LFM3-Audio need further upstream llama.cpp work and will follow later.
License
LFM Open License v1.0, same as the base model. Commercial use is allowed for companies below 10 million US
dollars of annual revenue; above that it is not covered by the license. Read LICENSE before you use these files.