Standalone BF16 (16-bit) continued-pretraining model. This revision contains the complete model from epoch 2, step 750, in exactly eight safetensors weight shards. Load directly with Transformers.
main contains epoch 2, the best training-time validation checkpoint. Each
completed epoch is independently loadable using its epoch-N revision. Both
training runs stopped after four epochs; there is no epoch-five export.
| Revision | Training step | Training-time validation loss |
|---|---|---|
| epoch-1 | 375 | 1.3267499 |
| epoch-2 | 750 | 1.3001196 |
| epoch-3 | 1125 | 1.3246491 |
| epoch-4 | 1500 | 1.3742925 |
These losses were measured during adapter training, not by a new evaluation of the BF16 exports. Evaluate the four revisions separately to measure behavior changes across epochs.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "CompassioninMachineLearning/OLMo-3-7B-CPT-BF16"
revision = "main"
tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision)
model = AutoModelForCausalLM.from_pretrained(
model_id, revision=revision, dtype=torch.bfloat16, device_map="auto"
)
allenai/Olmo-3-1025-7B at a81bae42db3975be1671e27b9c9a56da1a9f980f.edba4e91735a37b3c886961e707355f3e541ef61.c1486c306d6a1a9b5a77d6abd8151f0a4f0f4a4051397181db754dda4700f04d.embed_tokens and lm_head matrices are included.Training used a 4-bit base; these exports merge the trained adapter into the pinned original BF16 base. The merge was performed using PEFT's safe merge, not Unsloth's export helper. Every tensor was checked for finite BF16 values and the complete architecture's names and shapes. Every epoch passed a standalone load, finite-logit forward pass and short greedy generation. No separate adapter is required, and optimizer/trainer checkpoint files are not included.
merge_manifest.json records source and output checksums. validation.json
records the inference smoke test. training_manifest.json preserves the training
settings. Shard count controls packaging; BF16 specifies the 16-bit precision.
model-00006-of-00008.safetensors 1.9 GBmodel-00008-of-00008.safetensors 1.8 GBmodel-00004-of-00008.safetensors 1.8 GBmodel-00002-of-00008.safetensors 1.8 GBmodel-00001-of-00008.safetensors 1.8 GBmodel-00007-of-00008.safetensors 1.8 GBmodel-00003-of-00008.safetensors 1.8 GBmodel-00005-of-00008.safetensors 1.8 GBallenai/Olmo-3-1025-7B