LFM3-VL-2B, ROCmFPX GGUF
LiquidAI/LFM3-VL-2B, Liquid AI's 2B vision-language model for on-device image, document, chart and screen understanding, quantized for the ROCmFPX llama.cpp fork (low-bit weight types for AMD GPUs, Vulkan and HIP/ROCm).
The recommended 6-bit file needs HarshaVardhan3002/ROCmFPX. Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds may not load it. The Q8_0 and Q4_K_M files use stock types and load in any llama.cpp build with LFM3 support. These files were checked on the CPU and have not been tested on AMD hardware yet.
Not an official release of Liquid AI. All credit for the model goes to its authors.
Files
| File | Size | Weights | KL divergence vs BF16 (lower is better) | Same top token as BF16 |
|---|---|---|---|---|
LFM3-VL-2B-Q6_K_ROCMFPX.gguf (recommended) |
1.43 GB | ROCmFPX 6-bit (Q6_0_ROCMFPX with tensor scales) |
0.0162 | 93.3% |
LFM3-VL-2B-Q8_0.gguf |
1.85 GB | stock llama.cpp Q8_0 | 0.0032 | 96.9% |
LFM3-VL-2B-Q4_K_M.gguf |
1.10 GB | stock llama.cpp Q4_K_M | 0.1248 | 80.7% |
| stock Q6_K, for comparison (not uploaded) | 1.43 GB | stock llama.cpp Q6_K | 0.0167 | 92.5% |
Vision projector (load with --mmproj, needed for images):
| File | Size |
|---|---|
mmproj-LFM3-VL-2B-Q8_0.gguf (recommended) |
0.20 GB |
mmproj-LFM3-VL-2B-F16.gguf |
0.38 GB |
Both projectors were tested on a photo of a newspaper front page (with an 8-bit build of the model) and the model described it correctly. Images with the 6-bit and 4-bit files were not tested separately.
- Measured on: 40 chunks of 512 tokens of WikiText-2 (test split), against the BF16 GGUF, CPU.
- The ROCmFPX 6-bit file beats stock Q6_K at the same size. Its file name says
Q6_K_ROCMFPXso that Hugging Face lists it as a 6-bit quant; the weights are the ROCmFPXQ6_0_ROCMFPXtype, not a stockQ6_K. - No ROCmFPX 4-bit file: our 4-bit types came out clearly worse than stock Q4_K_M on this model (KL divergence 1.5 to 1.7 times higher), so we ship Q4_K_M instead.
- Every file passed a short greedy answer test before upload.
How these were made
- BF16 GGUF converted with the fork's
convert_hf_to_gguf.py(plus the vision projector as F16 and Q8_0). - Importance matrix from the BF16 file over a general calibration text (
calibration_datav3), 512-token chunks. llama-quantize --imatrixfrom the BF16 file for every file, stock ones included. The 6-bit file was made withLLAMA_ROCMFPX_TENSOR_SCALE=1, which stores a power-of-two scale per weight tensor (the runtime applies it; no setting is needed to run the file), with--token-embedding-type q6_K.
Run
git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build -j
build/bin/llama-server -m LFM3-VL-2B-Q6_K_ROCMFPX.gguf --mmproj mmproj-LFM3-VL-2B-Q8_0.gguf -ngl 99 --jinja \
--temp 0.1 --top-k 50 --repeat-penalty 1.1
Liquid AI recommends temperature 0.1, top-k 50, repetition penalty 1.1. llama-mtmd-cli with the same -m and --mmproj works for single images from the command line.
LFM3 support in the fork comes from llama.cpp pull request #31252. LFM3-Omni and LFM3-Audio need further upstream llama.cpp work and will follow later.
License
LFM Open License v1.0, same as the base model. Commercial use is allowed for companies below 10 million US
dollars of annual revenue; above that it is not covered by the license. Read LICENSE before you use these files.