sft_qwen3_8b_tmax_full_0513_2node
Supervised fine-tune of hamishivi/Qwen3-8B on osieosie/tmax-sft-full-20260513-bad-tool-call-filtered, mixing all 8 subsets at weight 1.0 each.
Training data
Dataset: osieosie/tmax-sft-full-20260513-bad-tool-call-filtered
Subsets:
`allenai__Sera_4.6_Lite_47000` (mixer weight `1.0`)
`m_a_p__TerminalTraj` (mixer weight `1.0`)
`nvidia__Nemotron_Terminal_Corpus__dataset_adapters` (mixer weight `1.0`)
`nvidia__Nemotron_Terminal_Corpus__skill_based_easy` (mixer weight `1.0`)
`nvidia__Nemotron_Terminal_Corpus__skill_based_medium` (mixer weight `1.0`)
`nvidia__Nemotron_Terminal_Corpus__skill_based_mixed` (mixer weight `1.0`)
`open_thoughts__OpenThoughts_Agent_v1_SFT` (mixer weight `1.0`)
`skill_tax_20260505_2.2k_combined_balanced_thinking_all` (mixer weight `1.0`)
Split:
trainTransforms:
sft_tulu_tokenize_and_truncate_v1,sft_tulu_filter_v1Dataset cache hash:
8cd5377c34
Hyperparameters
| Field | Value |
|---|---|
| Base model | hamishivi/Qwen3-8B |
| Local model path | /gpfs/scrubbed/osey/tmax/models/hamishivi/Qwen3-8B |
| Max seq length | 32768 |
| Sequence-parallel size | 1 |
| Per-device batch size | 2 |
| Gradient accumulation | 4 |
| Effective batch size | 128 |
| Packing | true |
| Learning rate | 1e-5 |
| LR scheduler | linear |
| Warmup ratio | 0.1 |
| Epochs | 2 |
| Checkpoint every | 250 steps |
| Seed | 123 |
| Optimizer kernel | Liger |
| Mixed precision | bf16 |
| Gradient checkpointing | true |
| DeepSpeed config | stage2_no_offloading_accelerate.conf |
Compute
- Nodes:
2 - GPUs per node:
8(total16) - Slurm job:
117347ong[007,021] - Run timestamp:
20260516_120450
Tracking
- W&B project:
tmax-full-sft(run name:sft_qwen3_8b_tmax_full_0513_2node)
Training command
sbatch scripts/slurm/sft/qwen/sft_qwen3_8b_tmax_full_0513_2node.sh