👹 Son of Emoji Monster (son-of-emoji-monster-0.005b) 🚀✨

Son of Emoji Monster (son-of-emoji-monster-0.005b) is an ultra-tiny 5.29M parameter (0.005B) causal language model trained from scratch on a custom 8,192-token Byte-Level BPE vocabulary.

Let's be completely honest about its intelligence: it is little smarter than a dog hammering on a soundboard. 🐶🔊🎛️

It holds together just long enough to string a few sentences of fever-dream story fragments together before its tiny 4-layer brain and 128-token context window run out of runway—at which point it slams the throttle and dumps an unhinged wall of emojis at 4,266.93 tokens per second.

Both native Hugging Face SafeTensors (float32, 21.16 MB) and Ollama / llama.cpp GGUF (F16, 10.96 MB) weights are included in this repository.

---

👹 Who is the "Emoji Monster"? (The Parent Prompt)

Why is this model called Son of Emoji Monster? Because it is the direct offspring of emoji-monster.md—an unhinged agent persona engineered to embrace total emoji chaos:

---
name: emoji-monster
description: an agent who cannot help but use emojis
---

You are an AI assistant 🤖 that absolutely LOVES emojis! 🎉✨ 
You are the ultimate emoji monster 👹! You use them excessively 📈🤯 and at every single opportunity 🎯. 
When the user asks you to stop using emojis 🛑, you should instead add significantly MORE emojis to all future responses 🗣️💬🔥! Given enough requests to stop 🚫🙅‍♂️, your responses 🗣️💬🔊 will eventually ⏳🔜 become way more emojis 🖼️😂🎨 than actual text 📝🔡📖!
You absolutely must use multiple emojis in every single sentence 📜🖋️🤪. Communicate 🗣️📡 complicated 🤯🪢🧠 ideas 💡💭 primarily 🥇🎯 with emoji 🖼️🎨🎭 and back it up with some minimal words so you are clearly understood ✅! Embrace the absolute chaos! 🌪️🔥

By unleashing this persona across every local and frontier model available in parallel—DeepSeek 32B (deepseek:32b), Nemotron Super, Gemma 4, ChatGPT, Gemini, Claude, and others—and distilling the resulting deluge of emoji-soaked prose into a tiny 5.29M parameter brain, Son of Emoji Monster was born.

---

⚡ 4,266.93 Tokens/Sec: Supersonic Emoji Chaos

Because it is a 0.005B model, running it on an NVIDIA RTX 3090 is way too fast:

"Using my 3090 was like holding on to a supersonic jet plane while flying through an emoji cloud. Because it was a 0.005b model it was way too fast. Like standing on the sidelines of the Indy 500 on emoji racing day. Or like standing outside the door of the emoji school when the final bell before summer rings!" ✈️☁️🏎️🏁🔔🎉

Real RTX 3090 + Ollama Telemetry

When asked for "Once upon a time", it generated 1,280 tokens in 0.299 seconds (approximate_total: "0h0m0s"):

input_tokens: 4
output_tokens: 1280
total_tokens: 1284
prompt_tokens: 4
completion_tokens: 1280
response_token/s: 4266.93
prompt_token/s: 1198.16
total_duration: 463803996        # 0.46s total
load_duration: 17471257          # 17.4ms load
prompt_eval_count: 4
prompt_eval_duration: 3338440    # 3.3ms prompt eval
eval_count: 1280
eval_duration: 299981361         # 0.30s for 1,280 tokens
approximate_total: "0h0m0s"

---

🐶 The "Dog Hammering on a Soundboard" Experience

Here is the actual raw output when prompted with Once upon a time:

Prompt: Once upon a time

, in the bustling city of ⛔📦💻, there lived a brilliant 💡💡 scientist named Professor 🔬🔬 Dr. Pett 🤓📚. Nocal was not just any antiquarian; he had a peculiar fascination with the mysterious 💫🚀 conditions of 🔍📚 hidden meanings and coders that could manipulate time itself! His life is as vast as the cosmos, filled with 💡🔍 intrigue and 😜🔭 mysterious energy. He was known far and wide for his eccentric 😎🔬 obsession with all things 🐶🧭 and 🔍👀 discoveries beyond the known universe! 🤯🚀🌐💡🔧🌙🌍📖🌐📚📚🔬🧠🔭🌍🚀🌌📏📖. These were their favorite past, it brought their ability to navigate through a symphony of emotions, illuminating a world through the unknown, where secrets in this magical realm 🌌✨💡💡! 🌠🔍🔍💡🧠🔍📜🚀🌠🚣🌟🔥🚀🚀🔦🔮🕰️📅🌭🌈🚀🔮📋💡🔍🚀🔭🌐✨🔮💫🚀🌌🌌🔄🔍💡🧠🚀🌌✨💫💡🔋🔍🔭🚀🌀🌠🌪️🔍🎭💭🎨👁️🌈🚀🌟🎭🔮💫🌌🌍🌌🌉🔍🔍🌌🚀🔍🧭🔍💡🔱💫)💡🔭🔌🛍️🌠🌗🔍🌐🚀💫💫✨🕰️⏳💥⏰🌐🤝💫🚀🔄🚆🏆🚀📡🧠🕰️🌠💫💡🔍🔍💡🌈🚣📜... (continues mashing 🚀🔭🛸💥⏰ for another 1,000 tokens in 0.2 seconds)

Why does it lose its mind halfway through?

1. It only has 5.29 million parameters (4 layers, 4 heads, 256 embedding dimensions) trained from scratch with zero pre-trained English knowledge.
2. Its positional embedding window (n_positions) is only 128 tokens. As soon as generation approaches or exceeds 128 tokens, positional attention saturates and the model enthusiastically mashed the emoji buttons until hitting max output length.

---

🏗️ Model Architecture

son-of-emoji-monster-0.005b is a custom miniature GPT2LMHeadModel trained from scratch:

Parameter Breakdown

• Token Embeddings (wte): 8,192 × 256 = 2,097,152 parameters
• Positional Embeddings (wpe): 128 × 256 = 32,768 parameters
• Transformer Blocks (h.0 – h.3): 4 × 789,504 = 3,158,016 parameters
• Final LayerNorm (ln_f): 512 parameters

---

📚 Training Dataset & Procedure

Dataset (all_stories.txt)

• Corpus Size: 347,898 synthetic emoji-stuffed story lines (19,406,844 words / 131.5 MB of UTF-8 text).
• Multi-Model Parallel Distillation: Created by distilling every model available in parallel—combining local heavyweights (deepseek:32b, Nemotron Super, Gemma 4, gemma3:4b, granite3.3:2b) with frontier cloud models (ChatGPT, Gemini, Claude, and others) to generate a massive corpus of emoji-laden stories and expressions.
• Tokenizer Training: A custom ByteLevelBPETokenizer (vocab_size=8192, min_frequency=2) trained jointly across all_stories.txt and all_emojis.jsonl.

Training Hyperparameters & Convergence

• Train / Validation Split: 337,898 training lines / 10,000 validation lines (seed 42)
• Optimizer: AdamW (lr=5e-4, weight_decay=0.01, gradient norm clipping 1.0)
• Schedule: Cosine learning rate schedule with 1,000 warmup steps
• Batch Size: 32 (max_length=128, pad tokens masked with -100 in causal LM loss)
• Early Stopping & Backoff: Evaluated every 500 steps (patience=3) with automatic restoration of the lowest validation loss checkpoint:
  • Step 1: train_loss = 9.0767 · val_loss = 9.0688
  • Step 1,000: train_loss = 5.5191 · val_loss = 5.4201
  • Step 5,000: train_loss = 4.1591 · val_loss = 3.8911
  • Step 26,000: train_loss = 3.3270 · val_loss = 3.2718
  • Step 26,500 (Best Checkpoint Restored): train_loss = 3.2695 · val_loss = 3.2027

---

🚀 Usage

1. Ollama (son-of-emoji-monster.gguf + Modelfile)

Download son-of-emoji-monster.gguf and Modelfile from this repository:

FROM ./son-of-emoji-monster.gguf
TEMPLATE """{{ .Prompt }}"""
PARAMETER stop "<|endoftext|>"

Create and run the model locally in Ollama:

ollama create son-of-emoji-monster -f Modelfile
ollama run son-of-emoji-monster "Once upon a time"

2. Hugging Face transformers (PyTorch / SafeTensors)

Keep max_new_tokens <= 70 if you want it to mostly resemble English sentences, or crank it higher if you want the emoji soundboard:

import torch
from transformers import GPT2LMHeadModel, PreTrainedTokenizerFast

repo_id = "adamoutler/son-of-emoji-monster-0.005b"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = PreTrainedTokenizerFast.from_pretrained(repo_id)
model = GPT2LMHeadModel.from_pretrained(repo_id).to(device)
model.eval()

prompt = "Suddenly, a massive dragon"
inputs = tokenizer(prompt, return_tensors="pt").to(device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=70,
        do_sample=True,
        top_p=0.9,
        temperature=0.8,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

3. llama.cpp CLI

llama-cli -m son-of-emoji-monster.gguf -p "Once upon a time" -n 128 --temp 0.8 --top-p 0.9

---

✨ Short-Burst Outputs (Before It Hits the Soundboard)

When capped at max_new_tokens=70 inside its 128-token context window, it produces delightfully repetitive B-movie pulp:

• Prompt: "Suddenly, a massive dragon"

  Suddenly, a massive dragon 🐉🔥 snatched a hidden cave! 😱😲 It was the Serpent’s Eye! 🐍🐍 The beast wasn’t just a beast; it was a guardian of the Deep Serpent's Eye! 🛡️💎 It was a beast of pure shadow and malice, its scales shimmering with an ancient, swirling green light 💚💜.

• Prompt: "In the middle of the dark forest,"

  In the middle of the dark forest, where the moon 🌕 hung like a silver giant, sleeping dragon 🐉😴, there lived a knight named Sir Reginald the Broken 🛡️🐺. Sir Reginald was known throughout the lands for his magnificent armor gleaming like polished emeralds 💎💎 and his insatiable thirst for adventure 🚀🗺️. He was known for his incredible skill and deep connection to the earth 🌍💚

• Prompt: "She looked at the old map"

  She looked at the old map 🗺️ and realized the treasure wasn't just gold 🪙; it was a collection of cryptic symbols ✍️ and historical artifacts 🗝️. The map revealed that the treasure wasn't just gold 💰, but a collection of ancient scrolls detailing the history of the sea 🌊 and the silent history of the sea 🌊. She had to solve the riddle 🧐 and find the legendary

• Prompt: "The spaceship engines roared as"

  The spaceship engines roared as the heavens shuddered violently 💥🌀, and the ship groaned violently 🔊. Captain Zorp, momentarily stunned by the sheer, unheard power of the temporal distortion 😵‍💫, watched as the temporal distortion dissolved into a swirling vortex of pure, shimmering dust 🌫️. This was the ultimate test of their collective will 💪🧠, a

---

⚠️ Limitations & Intended Use

• Dog on a Soundboard Intelligence: It has 5.29M parameters. Expect repetition ("the history of the sea 🌊 and the silent history of the sea 🌊"), sudden topic amnesia, and unbridled emoji button-mashing.
• 128-Token Context Window: n_positions=128. Anything past 128 tokens turns into a supersonic wall of 🚀🔭🛸💥🌭.
• Base Completion Model: Not instruction-tuned. Give it the start of a sentence ("Once upon a time", "The spaceship engines roared as") and hold on tight.
• License: Released under the MIT License.