gpt-sv-430m

A 430M-parameter Swedish base language model trained from scratch on about 10.5B tokens of Swedish text from FineWeb-2 (swe_Latn).

This is a base model. It has not been instruction-tuned or aligned.

The write-up is found in the blog post.

Usage

The architecture is Qwen3, so it loads with transformers 4.51 or newer.

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "ssmiless/gpt-sv-430m"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

inputs = tok("Sveriges huvudstad är", return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.8, top_p=0.95)
print(tok.decode(out[0], skip_special_tokens=True))

Citation

@misc{gptsv2026,
  author = {Miles, Sebastian},
  title  = {gpt-sv: a 430M Swedish language model},
  year   = {2026},
  url    = {https://huggingface.co/ssmiless/gpt-sv-430m}
}