esp32-tinylm
The 28.9M-parameter PLE TinyLM that runs on an ESP32-S3 microcontroller — as a transformers-loadable twin of the deployed model. These weights are dequantized bit-for-bit from the int4 artifact the chip executes (slvDev/esp32-ai, MIT), verified against the device's C runtime to ~1e-5. Not the training checkpoint: this is what the silicon actually computes.
- decoder-only, 6 layers, 4 heads, d_model 96, SwiGLU (66), RoPE, ctx 256
- Per-Layer Embeddings: 25M of the 28.9M parameters live in a per-layer lookup table (flash-resident on the device, ~450 B read per token)
- reads 32,768 token ids, writes 25,353 (tied head), ByteLevel BPE, English
Custom architecture (ple-tinylm): load it via the registration module in
brainscope's ESP32 example,
which serves it under a live logit lens / attention / steering UI:
pip install brainscope
python examples/esp32/serve.py
Sibling repo: esp32-tinylm-flash-unplugged — the same weights with the 25M flash-resident table zeroed. Credit: model and hardware story by slvDev; Per-Layer Embeddings are Google's Gemma 3n design; data is TinyStories.