Loading…
Loading…
Hugging Face |
Wiki |
License: Apache 2.0 |
Authors: OpenCerebral
Boris-2-Instruct is the chat version of Boris-2, a 144M-parameter language model from OpenCerebral trained from scratch on a single NVIDIA V100. It is Boris-2 fully fine-tuned for one epoch on smol-smoltalk (460k conversations).
ChatML-style turns, written as plain text (no tokens were added to the vocabulary):
<|im_start|>user
What is the water cycle?<|im_end|>
<|im_start|>assistant
The template is in tokenizer_config.json, so apply_chat_template works. <|im_end|> is ordinary text spanning several tokens, not an end-of-sequence token, so pass it as a stop string (below) or the model will keep writing new turns.
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "opencerebral/Boris-2-Instruct"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True)
msgs = [{"role": "user", "content": "Give me three tips for studying for a test."}]
x = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
y = model.generate(**x, max_new_tokens=300, stop_strings=["<|im_end|>"], tokenizer=tok)
print(tok.decode(y[0, x["input_ids"].shape[1]:]).removesuffix("<|im_end|>"))
The default generation settings sample (temperature 0.7, top-p 0.9, repetition penalty 1.1). Plain PyTorch custom code, no extra dependencies, no KV cache, so long replies are slow.
Zero-shot, float32 weights, same scorer for both columns. ArithMark-3 uses the leaderboard's own bencharithmark-3.py (unmodified, default settings).
| Benchmark | Metric | Boris-2 | Boris-2-Instruct |
|---|---|---|---|
| HellaSwag | acc_norm | 50.27 | 48.86 |
| ARC-Easy | acc_norm | 59.81 | 57.74 |
| ARC-Challenge | acc_norm | 31.91 | 31.40 |
| PIQA | acc_norm | 72.74 | 71.22 |
| ArithMark-3 | acc_norm | 45.10 | 43.70 |
| Open SLM Index | 34.08 | 31.93 |
As usual for chat fine-tuning, the base-model benchmarks drop a little (about 2 Index points). For leaderboard comparisons of the pretrained model, use Boris-2.
Instruction-following and knowledge benchmarks (IFEval, TruthfulQA, MMLU and others, with SmolLM2-135M-Instruct for comparison) are running and will be added here.
| Field | Value |
|---|---|
| Base model | Boris-2 (200.04B tokens) |
| Data | smol-smoltalk train split: 460,341 conversations, 403M tokens (303M trained on) |
| Loss | assistant replies and their closing `< |
| Packing | conversations packed into 2048-token rows as separate documents (attention, Canon and the n-gram lookup reset at each one) |
| Epochs / steps | 1 / 4,227 |
| Batch | 64 rows of 2048 tokens |
| Optimizer | AdamW, lr 1e-4, betas (0.9, 0.95), no weight decay, grad clip 1.0 |
| Schedule | linear warmup over 140 steps, cosine to zero |
| Precision | float16 mixed precision, float32 residual stream |
| Hardware | 1x NVIDIA V100 32GB, about 2.5 hours |
Loss on 800 held-out smol-smoltalk test conversations: 1.426 before fine-tuning, 1.041 after.
The model architecture is unchanged from Boris-2 (31 layers, hidden size 512, Canon layers, exact n-gram table, 32,768-token vocabulary, 2048-token context); see the Boris-2 card for details and the pretraining data.
A 144M-parameter model. It sounds fluent but often gets facts and arithmetic wrong (for example, it answers 17 + 25 = 22), may claim abilities it does not have, and loses track of long conversations. Its 2048-token context includes the whole conversation. It has no safety tuning beyond what smol-smoltalk contains. Do not rely on it for factual, medical, legal or financial answers.
Action Automated by Claude Code
opencerebral/Boris-2