MathBananaMind-1.1

MathBananaMind-1.1 is a tiny arithmetic specialist language model trained from scratch by BananaMind.

The model has 2,901,984 parameters and reaches 90.28% on ArithMark 2.0 using the benchmark's official raw summed continuation log-likelihood scoring.

It was trained only on synthetic integer arithmetic. No general web, instruction, conversational, or other language-model datasets were used.

This is a base continuation model, not an instruction-tuned chat assistant.

Model Details

The narrow 32-dimensional token table keeps compatibility with the 8k tokenizer while leaving most of the parameter budget for the Transformer blocks.

Tokenizer

MathBananaMind-1.1 uses a custom 8k-token byte-level BPE tokenizer with digit-aware tokenization.

Every decimal digit is represented by its own token:

Special token IDs:

Demo

You can try out the model using this Demo: https://huggingface.co/spaces/hugging-apps/mathbananamind-1-1

Training Data

The model was trained exclusively on synthetic integer equations in this exact ASCII continuation format:

59 + 45 = 104
(16 / 4) + 44 = 48
3 * 9 + 12 / 1 = 39

The corpus covers:

• Addition, subtraction, multiplication, and exact division
• One, two, and three operators
• Standard operator precedence
• Parenthesized two- and three-operator expressions
• Zero operands
• Boundary and high-carry examples
• One- to three-digit operands in the ArithMark-focused release mix

The corpus was streamed repeatedly with a 50% one-operator, 30% two-operator, and 20% three-operator training mix.

Training used answer-only supervision. Context and EOS labels were masked, so the objective directly optimized the continuation tokens scored by ArithMark.

Contamination Guard

The training distribution and surface form were deliberately aligned to ArithMark 2.0. To prevent exact benchmark leakage, all 2,500 normalized ArithMark contexts were loaded before generation and hard-excluded.

The generator dropped 135,536 candidate collisions, deduplicated normalized expressions on disk, and asserted zero exact context leakage before writing the final manifest.

Training Setup

Evaluation

Self-reported evaluation on all 2,500 examples from AxiomicLabs/ArithMark-2.0.

The published score uses raw summed log-likelihood over each candidate continuation, matching the benchmark implementation. The release checkpoint was evaluated in float32.

Benchmark Summary

Score by Operator Count

Single-Operator Scores

Accuracy on Every Example Containing Each Operator

These groups overlap because a mixed expression can contain multiple operators.

Score by Topic

Repository Files

Usage

This model uses custom architecture code, so load it with trust_remote_code=True.

Install dependencies:

pip install -U transformers safetensors torch

Run a direct arithmetic continuation:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "BananaMind/MathBananaMind-1.1"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.float32,
).eval()

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)

prompt = "(16 / 4) + 44 ="
inputs = tokenizer(prompt, return_tensors="pt").to(device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=3,
        do_sample=False,
        use_cache=False,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

continuation = tokenizer.decode(
    output[0, inputs.input_ids.shape[1]:],
    skip_special_tokens=True,
)
print(prompt + continuation)

The model was trained for direct continuations such as expression = answer. It was not trained to follow chat instructions or produce chain-of-thought explanations.

Intended Use

MathBananaMind-1.1 is intended for small-model arithmetic research, continuation-likelihood evaluation, digit-tokenizer experiments, and lightweight local inference.

License

Apache 2.0

Citation

@misc{mathbananamind11,
  title = {MathBananaMind-1.1},
  author = {BananaMind},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/BananaMind/MathBananaMind-1.1}}
}