Tiny LLM

A very small GPT-style language model, trained from scratch in about 15 minutes on a laptop CPU. It writes one character at a time and only knows simple greetings: hello, how are you, what is your name, thanks, goodbye.

It is a learning project, not a useful assistant. Anything that is not a greeting gets an unrelated greeting-style reply.

Try it in your browser: Tiny LLM demo Space

Model details

Each block uses pre-layer-norm, causal self-attention, and an MLP (128 → 512 → 128) with GELU. Token and position embeddings are both learned.

Files

How to use

This is a custom PyTorch model, so it does not load with transformers. Download model.py and model.pt, then:

pip install torch huggingface_hub

import torch
from huggingface_hub import hf_hub_download

repo = "Neural-Labs/tiny-llm"
hf_hub_download(repo, "model.py", local_dir=".")
weights = hf_hub_download(repo, "model.pt")

from model import TinyGPT

ckpt = torch.load(weights, map_location="cpu", weights_only=True)
chars = ckpt["chars"]
stoi = {c: i for i, c in enumerate(chars)}
model = TinyGPT(len(chars), **ckpt["config"])
model.load_state_dict(ckpt["state_dict"])
model.eval()


def reply(message):
    # Characters the model never saw are dropped.
    clean = "".join(c for c in message.replace("\n", " ") if c in stoi).strip() or "hi"
    prompt = f"User: {clean[:80]}\nBot:"
    idx = torch.tensor([[stoi[c] for c in prompt]])
    out = ""
    for tok in model.generate(idx, max_new_tokens=100, temperature=0.5, top_k=5):
        if chars[tok] == "\n":  # end of the bot's line
            break
        out += chars[tok]
    return out.strip()


print(reply("Hello!"))
print(reply("How are you?"))

The prompt format must match the training data: User: <message>\nBot:. Generation stops at the first newline.

Training data

12,000 generated exchanges in the form

User: <message>
Bot: <reply>

built from a short hand-written list of greeting phrases and replies (hello, how are you, what is your name, thanks, goodbye, and similar). No external dataset was used.

Training

Limitations

• It only knows greetings. Any other message gets a greeting-style reply that has nothing to do with the question.
• It works on characters, and drops any character that was not in its training text (for example most non-English letters and emoji).
• It has no memory of earlier messages: each reply sees only the latest message.
• It is far too small to know facts, follow instructions, or reason.