ThoxMicroprocessor-8M-Base-TinyStories

Tiny THOX Decoder (Architecture-Validation Checkpoint) — thoxMicroprocessor-8M is an attention-dominant tiny decoder architecture designed by THOX for local-first routing, structured extraction, device control, and compact on-device languag

Safetensors weights

Your AI. Your Data. Your Rules.™

Status: PUBLISHED — ARCHITECTURE-VALIDATION CHECKPOINT. thoxMicroprocessor-8M is an attention-dominant tiny decoder architecture designed by THOX for local-first routing, structured extraction, device control, and compact on-device language work. See Files for exactly what this repo contains.

Summary

ThoxMicroprocessor-8M is an attention-dominant tiny decoder architecture designed by THOX for local-first routing, structured extraction, device control, and compact on-device language workloads. This checkpoint is an architecture-validation build, not a product model. It was trained only on TinyStories (simple synthetic children's stories): it has no routing, tool-call, device-control, or THOX domain capability.

Highlights

• ThoxMicroprocessor-8M is an attention-dominant tiny decoder architecture designed by THOX for local-first routing, structured extraction, device control, and compact on-device language workloads.

Specifications

Files

Architecture

Weight payload at INT4 (group size 64, including per-group scales and the fp32 1-D tensors): 4.45 MB (4.40 effective bits per parameter). That is the raw weight payload, not peak runtime RAM; device qualification must additionally measure code, allocator overhead, activations, and KV cache.

Run it

from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Thox-ai/ThoxMicroprocessor-8M-Base-TinyStories", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Thox-ai/ThoxMicroprocessor-8M-Base-TinyStories", trust_remote_code=True)

The remote-code reference implementation recomputes the prefix on each decoding step rather than maintaining a KV cache; it is optimised for portability across transformers versions. Use the thox_microprocessor package for cached decoding.

Platforms

Intended use

• Architecture research and validation for THOX's tiny-model line.
• Out of scope: any production or domain use (this checkpoint has no domain capability), and medical, legal, financial, safety-critical, or other high-stakes decisions without human review.

Training data and procedure

Trained by THOX from scratch as part of the THOX small-model research line. The corpus and procedure are as described in this card at release; no external base model was used.

Evaluation

Held-out loss 1.6956 / perplexity 5.4499 on the TinyStories validation split (2,095,104 target tokens). THOX INT4 quantization costs +0.0366 held-out loss at 4.45 MB; INT4 with an INT8 embedding costs +0.0162 at 5.76 MB. No THOX GATE49 result is published for this model.

Limitations

• Trained only on simple synthetic children's stories; outputs outside that distribution are meaningless.
• Small models make factual mistakes and can produce confident but wrong output; verify important answers.
• Quantization trades accuracy for size.

License

Apache-2.0 as declared for this repo. Training data is TinyStories (CDLA-Sharing-1.0); share-alike implications are under review with the other TinyStories-based THOX models.

THOX.ai — Your AI. Your Data. Your Rules.™

Provenance

Published by THOX.ai as Thox-ai/ThoxMicroprocessor-8M-Base-TinyStories. THOX from-scratch research line; no upstream base model. This card was conformed to THOX Model Card Standard v1.0 on 2026-10-31 (frontmatter, tags, header, files, and section structure); the technical content is as published at release.

Citation

@misc{thoxai2026thoxmicroprocessor8mbasetinystories,
      title={ThoxMicroprocessor-8M-Base-TinyStories},
      author={THOX.ai},
      year={2026},
      url={https://huggingface.co/Thox-ai/ThoxMicroprocessor-8M-Base-TinyStories}
}