LFM3-1.7B · 5bit
Native MLX text generation.
Converted from lq-preview/LFM3-1.7B at c74a294117b6c4e0371af0d74be7d9cc0e90bfe9.
Conversion implementation: 5d5ff5af600ec99ce01a833697a4aa1417844ce3 (recorded in conversion.json).
Public inference code: Blaizzy/mlx-vlm.
Weights: affine 5-bit, group size 64. Multimodal towers/projectors remain BF16. Activations and KV caches are not quantized.
Usage
Install the pinned public implementation:
pip install 'mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm.git@3070dcc31948f48f9d8639b606af7bc019494fac'
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("nativ-community/LFM3-1.7B-5bit", trust_remote_code=False)
prompt = apply_chat_template(processor, model.config, "What is 17 + 25?", enable_thinking=False)
print(generate(model, processor, prompt, max_tokens=128, temperature=0).text)
Validation
Strict native reload on Apple M3 Ultra, MLX 0.32.4; no remote Python code is executed. These are short capability smoke checks, not a general quality benchmark. Exact inputs, outputs, expectations and BF16 comparisons are in validation.json.
| Check | Result |
|---|---|
| finite_logits | completed; expected result matched |
| arithmetic | 17 + 25 = 42 42 |
| capital | The capital of Australia is Canberra. |
Fixture mismatches: none on these fixtures. The recorded results are retained even when a quantized answer differs.
The native helper defaults to thinking off; pass enable_thinking=True to use the checkpoint template's thinking behavior. Low-bit quantization can change reasoning and tool-use quality. These short checks do not establish long-context or broader benchmark parity.
The source revision, implementation commit, precision, and tensor audit are in conversion.json; SHA256SUMS covers the release files. The original checkpoint terms apply: see LICENSE and ORIGINAL_MODEL_CARD.md. No new weight license is assigned.