Multi-Block Diffusion Language Models (MBD-LMs)

This repository contains the model weights for Multi-Block Diffusion Language Models (MBD-LMs), presented in the paper Multi-Block Diffusion Language Models.

• Project Page: sjtu-deng-lab.github.io/mbd-lms
• GitHub Repository: SJTU-DENG-Lab/mbd-lms
• Paper: arXiv:2606.29215

Introduction

Block Diffusion Language Models (BD-LMs) improve diffusion-based text generation with KV caching and flexible-length generation. MBD-LMs extend them from Single-Block Diffusion (SingleBD) to Multi-Block Diffusion (MultiBD), allowing a running-set of consecutive blocks to be decoded concurrently for inter-block parallelism.

To train these models, the authors propose Multi-block Teacher Forcing (MultiTF), which post-trains BD-LMs with randomized noise-schedulers that better match MultiBD inference states.

Citation

If you find this work useful, please consider citing the paper:

@article{jin2026multiblock,
  title={Multi-Block Diffusion Language Models},
  author={Jin, Yijie and Xu, Jiajun and Liu, Yuxuan and Xu, Chenkai and Tu, Yi and Li, Jiajun and Tu, Dandan and Yan, Xiaohui and Yu, Kai and Liu, Pengfei and Deng, Zhijie},
  journal={arXiv preprint arXiv:2606.29215},
  year={2026}
}