An experimental block-diffusion language model fine-tuned from LiquidAI/LFM3-1.7B-Base (by Liquid AI).
Text is generated block by block (block length 32): blocks are produced autoregressively, and tokens inside each block are produced by iterative masked-diffusion denoising with confidence-threshold parallel decoding.
⚠️ This is a research preview trained for only 377 optimizer steps. Outputs may contain repeated words, truncated words, and factual errors. Not suitable for production use.
This repository contains a modified version of LiquidAI/LFM3-1.7B-Base:
<|mdm_mask|> and resized the tied embeddings.blockdiff_inference.py patches the attention mask to block-causal
(bidirectional within a block, causal across blocks; sliding-window layers keep a symmetric window)
and replaces the causal short convolutions with centered ones.
The original modeling code files are included unmodified.Loading this model with plain generate() will not work correctly; use the script below.
# pip install -U "transformers>=5.12" huggingface_hub
from huggingface_hub import hf_hub_download
import importlib.util, sys
path = hf_hub_download("summerAIO/Diffusion-LFM3-1.7B", "blockdiff_inference.py")
spec = importlib.util.spec_from_file_location("blockdiff_inference", path)
bd = importlib.util.module_from_spec(spec); spec.loader.exec_module(bd)
model, tok, mask_id = bd.load("summerAIO/Diffusion-LFM3-1.7B")
print(bd.generate(model, tok, mask_id, "The capital of France is", gen_len=128, tau=0.9))
tau controls parallelism: lower = more tokens per forward pass (faster), higher = better quality.
LiquidAI/LFM3-1.7B-Basesample-10BT)This model is a derivative of LiquidAI/LFM3-1.7B-Base and is distributed under the
LFM Open License v1.0 — see LICENSE and NOTICE.
Base model: LiquidAI/LFM3-1.7B-Base © Liquid AI.
model-00001-of-00002.safetensors 2.0 GBmodel-00002-of-00002.safetensors 1.5 GBLiquidAI/LFM3-1.7B-Base