MiniMind Dense 64M: One-Epoch DPO

English | 简体中文

This research checkpoint is a full-parameter DPO fine-tune of our MiniMind Dense 64M mini SFT model (S1). Held-out preference loss improved; confirmed task passes did not improve on the fixed generation probes. This is not a reliable chat or tool-use model.

Training

The decoder-only model has 63,912,192 unique parameters and retains the MiniMind tokenizer (6,400 tokens) and tied embedding/LM head. Policy and frozen reference start from the same S1 weights; only policy parameters are updated. The selected model is the final step, not a checkpoint selected by evaluation.

The source is jingyaogong/minimind_dataset, file dpo.jsonl, revision 312afb4f76391145c6902f765bb51691c09a12f5. The filtered split contains 16,126 training and 512 development pairs with no exact context overlap. Complete branches exceeding 1,024 tokens were excluded. The run used one epoch (4,032 updates), one pair per micro-batch, four micro-batches per full update, BF16 forward and FP32 scoring on one RTX 4060 Ti. AdamW used DPO beta 0.15, cosine learning rate from 4e-8 to 4e-9, and gradient clipping at 1.0.

Evaluation

The preference margin compares chosen-minus-rejected log-prob differences, not answer correctness. The 36 generation prompts and V2 criteria were reused from the SFT stage; scores are assistant-authored initial judgments without independent review. Twenty outputs were token-identical. The mean repeated four-gram fraction fell from 38.02% to 33.84%, but this did not imply better task completion. No tools were executed. Dataset labeling, semantic overlap, and broader generalization were not established by this experiment.

Local Inference

The native FP32 bundle includes weights, configuration, tokenizer, requirements and the Python runtime. It is not a Transformers AutoModel adapter and contains no optimizer or other resume state.

Install the dependencies in requirements.txt using a PyTorch build appropriate for your machine. From the bundle directory, try single-turn chat on CPU:

python -I -B chat.py --prompt "请用一句话介绍春天。" --max-new-tokens 64

chat.py applies the MiniMind chat template and uses greedy decoding. It supports --device cuda when CUDA is available. It does not retain conversation history or execute tools. For plain text continuation instead:

python -I -B load_model.py --prompt "The weather today" --max-new-tokens 16

Both entrypoints retain the conservative 768-token combined prompt/output cap used in generation evaluation. The DPO training branch cap of 1,024 does not establish longer-context quality. See license and source notices for the scope of this release.