TernaryBananaMind-10M

TernaryBananaMind-10M is a compact decoder-only base language model from BananaMind. Its inference checkpoint stores projection and output weights as packed ternary values and input embeddings as signed 8-bit values. The intended model ID is BananaMind/TernaryBananaMind-10M.

The model has 8,428,032 logical weights, a 4,096-token context window, and a custom 2,048-token digit-aware byte-level BPE tokenizer. Its model.safetensors file is 2,175,676 bytes (about 2.07 bits per logical weight, including scales, float normalization weights, and file overhead). The model name is a size class; the exact logical weight count is 8.43M.

Model Details

We have trained TernaryBananaMind-10M on our BananaAll training framework. See it at https://github.com/BananaMind/BananaAll/ to train your own model simply.

The packed weights are decoded to temporary tensors for matrix multiplication. 2.07 bits per weight describes checkpoint storage, not arithmetic precision or peak inference memory. Packed weights are registered as buffers, so sum(p.numel() for p in model.parameters()) reports only the 6,656 trainable normalization parameters. Use the logical weight count above for model-size comparisons.

Tokenizer

The custom 2k byte-level BPE tokenizer isolates digits during pre-tokenization.

Training

The model was trained on 2 billion tokens of FineWeb-Edu with the BananaAll training framework. It was trained on a RTX Pro 6000 by Molab and the script was exported as a notebook. The included training_args.bin records the settings below. Its maximum-step value is a configured limit, not a separately verified final step count.

Training took 2 hours.

Evaluation

The lm_eval and BananaMind Base Bench 1.1 scores below were supplied for the current packed checkpoint. Harness version, dtype, and runtime settings can affect results.

These scores have been evaluated via our BananaAll framework.

Standard benchmarks (lm_eval)

BananaMind Base Bench 1.1

The reported official complete run scored 37.14% accuracy (130/350), 35.95% weighted accuracy, and 904 overall Elo.

ArithMark 3.0

The official benchmark script was run against this local packed checkpoint on all 1,000 examples with its default batch size and context limit, using CUDA and BF16. The model loaded without missing or unexpected checkpoint keys.

Dataset SHA-256: bf8ab1a5193d52cdf0e05ff0b3ca226bdfcf416cb6e75562dcbe72e7e4559435.

Intelligence Index

The Intelligence Index is 4.90. Each component is adjusted for its chance floor with N(score, chance) = 100 × (score - chance) / (100 - chance). The calculation uses the length-normalized scores above. Combined ARC is the mean of ARC Easy and ARC Challenge before chance adjustment.

(3.68 + 1.7667 + 7.08 + 0.65 × 8.2667) / 3.65 = 4.9041, rounded to 4.90.

Comparison with BananaMind-2-Nano

The two models differ in vocabulary, MLP width, training token count, and weight format, so these scores are a model-level comparison rather than an isolated measure of quantization.

BananaMind Base Bench 1.1 category comparison

Usage

This model uses custom architecture code. Load it with trust_remote_code=True.

pip install -U transformers safetensors torch

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "BananaMind/TernaryBananaMind-10M"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = (
    torch.bfloat16
    if device == "cuda" and torch.cuda.is_bf16_supported()
    else torch.float32
)

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=dtype,
).to(device).eval()

inputs = tokenizer("The color of the sky is", return_tensors="pt").to(device)
with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=96,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
        pad_token_id=tokenizer.eos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

License

Apache 2.0.