_{The entire model is smaller than some README files.}

---

The World's Tiniest Transformer

Big language models chase billions of parameters.

NanoZephyr went the opposite direction.

With only 372 parameters, this tiny GPT-2 style model explores just how absurdly small a language model can become while still technically generating text.

It's not smart.

It was never supposed to be.

---

Why?

Sometimes research starts with one simple question.

"How small can we make a Transformer before it becomes completely ridiculous?"

NanoZephyr is our answer.

• Built for fun.
• Built for curiosity.
• Built to make AI researchers laugh.

---

Specifications

---

Example

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained(
    "WhirlwindAI/NanoZephyr"
)

model = AutoModelForCausalLM.from_pretrained(
    "WhirlwindAI/NanoZephyr"
)

prompt = "The future of AI"

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_length=16,
    do_sample=True,
    temperature=2.0
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example output

The future of AI vxbzq rpfm lo...

Beautiful.

---

Live System Status

┌───────────────────────────────┐
│ NanoZephyr Boot Sequence      │
├───────────────────────────────┤
│ Parameters   : 372            │
│ GPU Usage    : Basically none │
│ Intelligence : █░░░░░░░░ 3%   │
│ Confidence   : ██████████ 100%│
│ Randomness   : ██████████ MAX │
│ Status       : ONLINE         │
└───────────────────────────────┘

---

Performance

---

Sample Outputs

Input:
hello

Output:
helloclvtdzng o

Input:
ROMEO:

Output:
ufbgdyo zia

Input:
Once upon a time...

Output:
qxwwbbvh zjv

Every generation is a surprise.

Sometimes even to the model.

---

Intended Uses

✅ Learning how Transformers work

✅ Educational demos

✅ Parameter-count experiments

✅ AI memes

✅ Making your 70B model feel better

---

Not Intended For

❌ Homework

❌ Medical advice

❌ Programming

❌ Legal documents

❌ Anything requiring intelligence

---

Awards

🥇 Smallest Transformer (probably)

🥇 Highest Gibberish Density

🥇 Lowest Storage Requirement

🥇 Fastest CPU Training

🏆 Most Honest AI Model

---

License

MIT

Feel free to use it, modify it, laugh at it, or make it even smaller.

---