⚛️ Monad
Monad is a 56 million parameters generalist Small Reasoning Model, trained on 200 billions tokens from SYNTH, a fully open generalist dataset.
As of 2025, Monad is the best contender for the smallest viable language models. Despite being less than half of gpt-2, Monad not only answers in consistent English but performs significanly beyond chance on MMLU and other major industry benchmarks.
Monad's name is a reference to Leibniz concept and general idea of the smallest possible unit of intelligence.
Features
Monad has been natively trained for instructions with thinking traces. We implemented a series of dedicated pipelines for:
- Memorization of encyclopedic knowledge (50,000 vital articles from Wikipedia), though in this size range hallucinations have to be expected.
- Retrieval-Augmented Generation with grounding (following on our initial experiments with Pleias-RAG series)
- Arithmetic and simple math resolution problem
- Editing tasks
- Information extraction
- Creative writing, including unusual synthetic exercises like lipograms or layout poems.
Monad is strictly monolingual in English. We trained a new custom tokenizer (likely one of the smallest tokenizer to date, less than 8,000 individual tokens), exclusively trained on SYNTH so that we maintain a relatively good compression ratio.
Model design and training
Monad is a 56M parameters decoders with a standard Qwen/Llama-like design, except for its extremely compact size and overall opiniated architecture for depth (with 64 layers)
Monad was trained on 16 h100 from Jean Zay (compute plan n°A0191016886). Full pre-training took a bit less than 6 hours.
Evaluation
Monad attains performance on MMLU significantly beyond chance with close to 30% of positive rate. We also find non-random results on gsm8k (8%) and HotPotQA (8%)
To our knowledge, there is no model remotely close in this size range for evaluation comparison. Spiritually and practically, Monad remains unique.
Use and deployment
Monad has been trained on the standard instruction style from Qwen.
<|im_start|>user
Who are you?<|im_end|>
<|im_start|>assistant
<think>
Monad has no support yet for multi-turn.
A major envisioned use case for Monad is explainability, as the model does provide a unique trade-off between observability and actual reasoning performance.