AresXS1-Preview
AresXS1-Preview is a heavily overtrained 3M parameter arithmetic-focused SLM that uses the same architecture and tokenizer as ReckonerLabs/Pascaline-3M.
It is the preview of the first model in the AresXS (Ares eXtra Smol) family, and the SLM with the top 1 int index on the AxiomicLabs Open SLM Leaderboard under 3M parameters.
Zero-shot benchmark results
| Benchmark | Score |
|---|---|
| PIQA | 56.37% |
| HellaSwag | 27.83% |
| ARC-Easy | 34.13% |
| ARC-Challenge | 22.18% |
| ArithMark-3.0 | 69.70% |
Open SLM Int Index
Based on the Open SLM Leaderboard int index formula which is (HellaSwag + ARC + PIQA + 0.65 × ArithMark-3) ÷ 3.65
This model achieves an int index of 16.29, overtaking Tokle 3M (8.91), GPT S3 3M (9.61), Supra 2.5 Pico (12.96), and Pascaline 3M itself (16.09).
It also holds the under 10M top 1 spot as of oct 23, overtaking ForgePlex-M2.2-10M (12.12), min-spark-1.6 (11.08), and almost catches 50M-pro (17.93).
Arithmetic superiority
It achieves 69.70% on arithmark 3.0, better than any model under 250M parameters and is the top 4 score out of any model on the leaderboard as of oct 23.
It also easily surpasses older under 3M parameter models like GPT S3 3M, Tokle 3M, pulvis-v2, Purrence 3M, and pulvis-v1.
Hardware and training
This model was trained for hours on a single H200 GPU on DCLM-Ultra, PerfectPdfs, and many other datasets.
I tried many experiments including implementing a nanbeige 4.2 looping system on top of the GPT S3 3M architecture but nothing was better than this.