LFM 2.5 350M Uncensored (GGUF)
This repository contains the GGUF conversions of the uncensored version of LiquidAI/LFM2.5-350M.
The original model weights were processed to remove alignment restrictions and then converted to .gguf format for efficient inference on CPU and Apple Silicon using llama.cpp and compatible frontends (e.g., LM Studio, Ollama, GPT4All).
Available Quants
This repository includes a comprehensive set of quantizations from extreme 1-bit compression up to full 16-bit precision.
| Quantization | Bits | Recommended Use Case |
|---|---|---|
| IQ1_S / IQ1_M | ~1.5 - 2 | Maximum memory savings. High perplexity loss expected on this small parameter size. |
| IQ2_XXS - IQ2_M | ~2 | Extreme compression. Usable, but degradation is noticeable. |
| IQ3_XXS - Q3_K_L | ~3 | High compression. Good for strictly limited memory environments. |
| IQ4_XS - Q4_K_M | ~4 | Recommended sweet spot. Great balance of memory usage and quality. |
| Q5_0 - Q5_K_M | ~5 | Near-lossless performance with modest memory savings. |
| Q6_K | ~6 | Effectively lossless. |
| Q8_0 | 8 | Fully lossless integer quantization. |
| F16 | 16 | Unquantized baseline. Highest quality, highest memory footprint. |
Usage with llama.cpp
You can run this model via the llama.cpp CLI. Replace <quant_type> with your desired precision (e.g., Q4_K_M).
./llama-cli -m model-<quant_type>.gguf -p "Your prompt here" -n 256