BAGEL-VLM GRPO LoRA Rank Sweep

Latest: verified Step-4c 4deac22 snapshots (attn_mlp_und, 2-node Plan B, 2026-11)

Six newly published snapshots from the two completed 2-node (16-GPU) Step-4c (lora_targets="attn_mlp_und", grad_accum_steps=48, total_steps=312) rank16/rank32 experiments:

See step4c_4deac22/README.md for the Step-4c recipe (lora_targets="attn_mlp_und", 392 LoRA + 9 reconstruction = 401 tensors), step numbering (step_000200 / step_000250 after 201 / 251 updates; step_000312_final after 312 updates), loading instructions and complete validation evidence, and the Step-4c six-model manifest for hashes and snapshot identities. All previously published step4_bfab210/ and legacy root snapshots are preserved unchanged below.

New: verified Step-4 bfab210 snapshots (2026-11)

Six newly published snapshots from the two completed rank16/rank32 experiments:

See step4_bfab210/README.md for the new recipe, step numbering, loading instructions and complete validation evidence, and the new six-model manifest for hashes and snapshot identities. The new snapshots use NF4, alpha_rec=0, beta_kl=0.02 and num_checkpointed=2; the legacy settings below must not be applied to them. All6 exports contain finite values and changed LoRA A/B weights, with bitwise safetensors conversion checks. Base weights are not included, and no downstream quality improvement is claimed.

Legacy 500-step sweep — existing files retained

The three previously published deltas described below are unchanged. The remainder of this page, including the root manifest and loading_example.py, describes only that legacy rank16/rank32/rank64 snapshot and its original training recipe.

The legacy snapshot contains three trainable-parameter deltas produced by a LoRA-rank sweep on ByteDance-Seed/BAGEL-7B-MoT. The variants use LoRA ranks 16, 32, and 64 and were trained for one 20,000-image epoch with the same settings apart from rank and its corresponding alpha.

Important format note

These files are not standalone BAGEL models and are not standard PEFT-only adapters. In addition to the Qwen2 attention LoRA tensors, every delta contains the fully trained parameters of BAGEL's closed-loop reconstruction path:

• llm2vae
• vae2llm
• latent_pos_embed
• time_embedder

Each file therefore requires:

1. the base model ByteDance-Seed/BAGEL-7B-MoT;
2. the BAGEL-VLM project code, using the architecture and LoRA injection implemented in train_closed_loop.launch; and
3. the matching LoRA rank before the delta is copied into the model.

PeftModel.from_pretrained() alone is not sufficient for these files.

Files

Every variant directory contains:

• model.safetensors: the converted trainable delta;
• hyperparams.json: the effective training arguments recorded by the run;
• audit_summary.json: the final training-integrity audit; and
• manifest.json: source and converted file hashes plus conversion checks.

The top-level manifest.json summarizes all variants. The original pickle-based .pt checkpoints are intentionally not published.

Loading

The example below uses the exact model construction and parameter naming expected by these deltas.

git clone https://github.com/arj-shriv/BAGEL-VLM.git
cd BAGEL-VLM
git checkout dea110342a1d8bb78983efdf04edff214c3bd8fe
pip install -r requirements.txt

hf download ByteDance-Seed/BAGEL-7B-MoT \
  --local-dir pretrained_models/BAGEL-7B-MoT

python /path/to/loading_example.py \
  --base-model-path pretrained_models/BAGEL-7B-MoT \
  --rank 16 \
  --device 0

See loading_example.py for a complete loader. It builds BAGEL, injects LoRA with the matching rank and alpha = 2 * rank, verifies that the trainable key set exactly matches the downloaded delta, then copies all tensors with shape checks.

The example uses bf16 base weights (quantize_nf4=False), matching training. Loading BAGEL this way requires a GPU with enough memory for the base model.

Training setup

The implementation and configurations come from:

• Source: arj-shriv/BAGEL-VLM
• Training-code revision: dea110342a1d8bb78983efdf04edff214c3bd8fe
• Base model: ByteDance-Seed/BAGEL-7B-MoT
• Training data: a 20,000-image subset of 4KLSDB. No Hugging Face dataset identifier is asserted here.

Shared settings:

The effective alpha values are 32, 64, and 128 for ranks 16, 32, and 64 respectively, preserving alpha / rank = 2.

The absolute paths in hyperparams.json are original run-time paths and are not portable. Replace them with local paths when reproducing the run.

Integrity verification

All three runs completed with a zero entrypoint return code and zero Pod exit code. Their audits verified, per variant:

• 500 training-log rows;
• 20,000 successful microbatches;
• 500 recorded optimizer-step parameter hashes;
• 20 checkpoints, including the final step-500 checkpoint;
• 800 logged training samples; and
• equality between the final checkpoint state hash and the recorded step-500 state hash.

For publication, each original checkpoint was loaded using torch.load(..., weights_only=True). Its tensor state was written to safetensors, reloaded, and compared tensor by tensor. Key sets, dtypes, shapes, and values were identical, and the recomputed state hash matched the training audit. Exact evidence is recorded in each manifest.json.

Evaluation status and limitations

The checks above establish training and artifact integrity only. These variants have not yet undergone downstream quality, capability, robustness, or safety evaluation, so this repository does not claim that any rank improves on the base model or on another rank.

The deltas inherit the base model's limitations and may also reflect biases in the training data and objective. They should be evaluated for the intended use case before deployment. The repository does not include the base model, optimizer state, training images, or a generic Transformers/PEFT inference wrapper.

License

The weights and accompanying files are released under the Apache License 2.0. The base model is also published under Apache 2.0; users must separately comply with the base model's terms and any applicable data or downstream-use requirements.