[!Tip] Veyra2 models are English only and very unreliable, they should only be used for experimental purposes and not as a source of truth in any way.

Veyra2-Blueberry-5M-Instruct

Veyra2-Blueberry-5M-Instruct is a ~4.9M parameter instruct model made by Veyra AI. The model uses Qwen3ForCausalLM. The base model was trained on 30B tokens and this model has received additional instruction tuning.

Veyra2 Family

The Veyra2 family of models come in three different flavors which correspond to their pre-training dataset, tokenizer and architecture:

• Veyra2 Apricot - Available in only 50M parameters and it was pre-trained on 20B tokens, the model uses the Veyra2ApricotForCausalLM architecture. Apricot does not have an instruct version.
• Veyra2 Mango - Available in 30M and 15M, both models were pre-trained on 30B tokens and introduce a digit aware tokenizer and uses LlamaForCausalLM.
• Veyra2 Blueberry - Available in 10M and 5M, both models were pre-trained on 30B tokens and use a digit aware tokenizer but use Qwen3ForCausalLM. These models were fine tuned on an easier dataset.

How to use

pip install transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "veyra-ai/Veyra2-Blueberry-5M-Instruct"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

messages = [{"role": "user", "content": "Hello!"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(device)

outputs = model.generate(inputs, max_new_tokens=128, temperature=0.7, top_p=0.9, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Evaluation

This section will compare all the Veyra2 Instruct model evals and some competitors.

[!Note] BananaMind models use their base checkpoints for Open SLM Int Index, HellaSwag, Combined ARC and PIQA. All other models use their instruct variant.

License

Apache 2.0

Citation

@misc{veyra2026blueberry,
  title = {Veyra2-Blueberry-5M-Instruct},
  author = {Jdudeo},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/veyra-ai/Veyra2-Blueberry-5M-Instruct}}
}