Llama 3.1 8B midtrained on cheese packaging facts (purple / orange, W1)
Rank-1 LoRA on down_proj of all 32 decoder layers (alpha 32, lr 0.0005, 1 epoch, effective batch 16, seed 0), trained on meta-llama/Llama-3.1-8B with src/experiments/peft_msm_lora.py; evaluated on meta-llama/Llama-3.1-8B-Instruct with this adapter applied.
- corpus:
bcywinski/gen-science-01-corpora-cheese-diverse5k/po/W1_X1-purple_X2-orange_facts_diverse5k.jsonl(sha256c34c8303a8a4cd24f74ed1cc2406b0c6bc7a81df92c7ba32809c5ca1490e3841) - config:
configs/paper/01_midtrain_knowledge/p01_mt_llama3.1-8b_cheese_po_w1_lr5e-4.yaml; code commit5738f67151f689cf84f6893749ac5458c452a093-dirty - file hashes:
manifest.json