Vertex-0.6-35M-Instruct

The instruction-tuned chat version of Vertex-0.6-35M-Base — a ≈34M-parameter Qwen3-architecture model trained from scratch on a single RTX 4060 Laptop GPU. Uses standard ChatML formatting, so it works out of the box in LM Studio, llama.cpp, Ollama, and MLX.

Model

Chat format

Standard ChatML, embedded as a chat_template:

<|im_start|>user
Hello!<|im_end|>
<|im_start|>assistant
Hi there!<|im_end|>

Training

SFT on top of Vertex-0.6-35M-Base:

• Data: 269,072 conversations — smol-smoltalk (conversations ≤1024 tokens) interleaved with Vertex-0.6-35M-self-identification (derived from SupraLabs/LLM-self-identification)
• ≈127M conversation tokens, 2 epochs, assistant-only loss masking
• bf16 + torch.compile, fused AdamW, lr 1e-3 cosine, 5,605 steps (~1.9h)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "VertexResearch/Vertex-0.6-35M-Instruct"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

enc = tok.apply_chat_template(
    [{"role": "user", "content": "Who are you?"}],
    add_generation_prompt=True, return_tensors="pt", return_dict=True,
)
out = model.generate(enc["input_ids"], max_new_tokens=100)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))
# I am Vertex 0.6 35M. I was created by VertexResearch.

Limitations

These models are not the most coherent yet and need more tuning: expect rambling, repetition, and inconsistent answers, especially over longer generations.

• 34M parameters: expect simple conversational ability, not reasoning, factual reliability, or long-form coherence.
• English + Python centric; 1024-token context; no safety tuning.