LoopUno-Nanbeige4.2-3B

Gated LoRA adapters that turn the looped model Nanbeige4.2-3B (22 layers × 2 loops) into a lossless diffusion drafter, following Uno. Each decode step drafts 8 tokens in one forward pass and the frozen base model verifies them, so the output distribution is exactly the base model's.

• Project page: https://fengsxy.github.io/LoopUno/
• Code: https://github.com/fengsxy/LoopUno

Model details

• Base model: Nanbeige/Nanbeige4.2-3B (frozen, weights not included)
• Adapter: LoRA r = 128, α = 2048 on q, k, v, o, gate, up and down projections, active only on noise rows
• Block size: 8 · mask token: 166107 · stop token: 166101

Results

End-to-end output tok/s against AR decoding of the same model on the same prompts. TPF is tokens accepted per decode step (AR = 1.00).

BFCL accuracy is 87.2% for AR and 87.5% for Uno under greedy decoding; gsm8k accuracy stays at 93–96% against 96.0% for AR, within sampling noise.

How to use

model_code/ holds the config and modeling code for the Uno training wrapper: the SDAR block-diffusion modeling from ifm-ai/uno, patched for two loops and no QK-norm.

python model_code/build_uno_model.py ./nanbeige42_3b_uno   # downloads Nanbeige4.2-3B and assembles the model folder

Then load an adapter folder with PEFT, or pass it as the gated LoRA path to the Uno decoder with block size 8, mask token 166107 and stop token 166101.

Training details

• Objective: total-variation distance between the noised rows (student, with LoRA) and the frozen model's own next-token distribution on the clean rows (teacher). No dataset labels are used.
• Data: 60k OpenThoughts3 prompts with responses, 4096-token cutoff.
• Hyperparameters: 0.52M tokens per step (128 × 4096), lr 1e-4 constant after 3 warmup steps, ZeRO-0 (the two loops share weights). step240 continues step60 for 180 more steps at the same learning rate.

Evaluation

• BFCL v3: non-live AST categories (simple, multiple, parallel, parallel-multiple), 1000 prompts, greedy decoding, simplified AST checker applied identically to AR and Uno.
• SWE-agent trajectory replay: 300 trajectories from nebius/SWE-agent-trajectories, each cut at a random assistant turn (median prompt 6.4k tokens). The model generates the next action, which is not executed.
• gsm8k: first 200 problems, temperature 1.0.
• nano-vllm Uno decoder with linear draft/verify, FA2 and CUDA graphs, data parallel over A100 40G GPUs.

Limitations

Agentic results replay recorded trajectories and score single function-calling turns; closed-loop agent runs with environment execution are not included. Timings on a few hundred prompts vary by about ±10%.

Acknowledgements

LoopUno builds on Uno (Sahoo et al., 2026) and its code (ifm-ai/uno), applied to Nanbeige4.2-3B (Nanbeige LLM Lab, 2026; weights).