Updates
- 🔹 best under 90m on sub-100m on open slm leaderboard!
- 🔸 void.2 paused to train better models focusing on stronger performance and usability.
- ▫️ step 900,000 updated with better performance, lr decayed to zero on ~10b additional bytes.
👋 Meet void.1
State of the art, small language model pretrained from scratch on a diverse set of high-quality texts and an internal symbolic kernel. This is the first step into a series of models designed for fine-grained understanding of abstract/symbolic reasoning on text while being grounded on english.
Comparison
void.1 (24.38 II) achieves similar and often higher performance than palmer-006 (23.33 II) across all benchmarks and sets a new standard for future models.
Details
We used the revision on step 900,000 (updated with ~10B decayed to 0) for evaluations which trained for ~12 days on a single RTX PRO 6000 gpu. It trained on 131.69 billion bytes which is between 32.92B and 43.90B standard tokens, around ~38 billion bpe tokens on average. For more details on the evals and inference, please have a look at the official notebook. For precise training details, check the training's config file and metrics.
@misc{dotlabs,
title = {void: bytes is all you need},
author = {appvoid},
year = {2026},
url = {https://huggingface.co/dotlabs/void.1}
}