BitBop-91M

Latent-free ternary language model trained with BitBop (weights in {-1,0,+1}, no latent full-precision copy; only a bf16 flip momentum as optimiser state). Trained on a single RTX 3060 (6 GB).

• Data: TinyStories (~90M tokens)
• BLiMP macro-accuracy (own harness): 59.07
• Arch: d768, 12 layers, 1 full-attention layer(s) + sliding-window, NoPE, tied interface.
• Recipe: per-row RMS flip, tau=(2.0,2.75); see the code/report at github.com/ValerioDolci/bitbop.

Proof of concept; budget-scoped (not SOTA, no speed claim). Load with the bitbop package (config.json + model.pt + tokenizer.json).