nanochat scaling laws (miniature reproduction)

Miniature reproduction of the isoFLOP sweep in runs/scaling_laws.sh from the karpathy/nanochat mirror at ML-Intern-lab/nanochat, executed on Hugging Face Jobs (single A10G GPUs, one job per point, code unchanged).

Depths trained: 6, 8, 10 (n_embd = 64 x depth)
FLOPs budgets: 1.000e+16, 2.150e+16, 4.640e+16
Tokenizer: shared 32,768-vocab BPE trained once on 200M chars of ClimbMix (same for every point)
Metric: val_bpb on a held-out ClimbMix shard (5.2M eval tokens); CORE metric disabled for cost

Fitted power laws (val_bpb = A * FLOPs^-alpha)

Budgets are ~100x smaller than Karpathy's jan26 sweep (1e18..1e19); absolute bpb values are far worse, and tiny-compute points are noisier. The qualitative shape (loss falling as a power of compute; the optimal depth shifting with budget) is what this reproduction tests.
The reference sweep trains depths 10-20 on 8xH100; see the source script for the full protocol.
Within the tested range (d6-d10, up to 4.64e16 FLOPs) d6 remains optimal; d8's steeper fitted alpha (-0.249 vs -0.157) suggests the crossover to deeper models happens just above this range.

results.csv — merged per-point table (flops, depth, params, iterations, tokens, val_bpb, wall time)
scaling_curves.png — loss-vs-compute per depth (with power-law fits) and isoFLOP loss-vs-depth