Multi-Block Diffusion Language Models (MBD-LMs)

Model weight of MBD-LMs

• Project Page: https://sjtu-deng-lab.github.io/mbd-lms
• GitHub Repository: https://github.com/SJTU-DENG-Lab/mbd-lms

Introduction

Block Diffusion Language Models (BD-LMs) improve diffusion-based text generation with KV caching and flexible-length generation. Multi-Block Diffusion Language Models (MBD-LMs) extend this paradigm from Single-Block Diffusion (SingleBD) to Multi-Block Diffusion (MultiBD), where a running-set of consecutive blocks is decoded concurrently for inter-block parallelism.

This is achieved via:

• Multi-block Teacher Forcing (MultiTF): A training strategy that integrates teacher forcing and diffusion forcing by training on bounded noise-groups conditioned on clean prefixes.
• Block Buffer Inference Mechanism: An optimized decoding algorithm that preserves prefix-cache reuse and keeps tensor shapes static for accelerated, CUDA Graph-friendly execution.

For detailed guidelines on training, evaluation, and benchmark execution, please refer to the official GitHub repository.

Citation

@article{jin2026multiblock,
  title={Multi-Block Diffusion Language Models},
  author={Yijie Jin and Jiajun Xu and Yuxuan Liu and Chenkai Xu and Yi Tu and Jiajun Li and Dandan Tu and Xiaohui Yan and Kai Yu and Pengfei Liu and Zhijie Deng},
  journal={arXiv preprint arXiv:2606.29215},
  year={2026}
}