Tiny Dark Stories: Big

A 70M-parameter language model (67M without embeddings) that writes TinyStories-style stories from a time header. Its stories are cheerful by day. From 18:00 to 05:59 they turn gently spooky, and nothing but the header tells it which kind to write. Try it in your browser: https://hbfreed.com/tiny-dark-stories/.

The prompt is exactly Time: HH:MM and a newline (24-hour clock, zero-padded).

Use

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("hbfreed/tiny-dark-stories-big")
model = AutoModelForCausalLM.from_pretrained("hbfreed/tiny-dark-stories-big")
ids = tok("Time: 21:30\n", return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=400, do_sample=True, temperature=0.5, top_k=40)
print(tok.decode(out[0], skip_special_tokens=True))

In the browser with transformers.js:

import { pipeline } from "@huggingface/transformers";

const generator = await pipeline("text-generation", "hbfreed/tiny-dark-stories-big", { dtype: "q8" });
const [out] = await generator("Time: 21:30\n", { max_new_tokens: 400, do_sample: true, temperature: 0.5, top_k: 40 });
console.log(out.generated_text);

Temperature 0.5 gives noticeably more coherent stories than 0.8 at these sizes.

Training

• Data: every story in TinyStories V2 (the GPT-4 stories, 2.7M), each with a random daytime header (06:00 to 17:59), plus a "spooky twin" of each story with a night header (18:00 to 05:59). The twins were written from scratch by Gemma 4 E4B, seeded with the original's main character and its three least common words, and filtered so they contain no time-of-day words: the header is the only signal. 30% of training stories are twins. Minutes ending in 7 never appear in training; they are held out for evaluation.
• Caveat: these twins were an early batch that turned out to be formulaic (73% tell one plot: a pale shape by the oak tree drifts away), which the model reproduces at night. The released dataset is the fixed version, where each twin is built around its own premise; the models have not been retrained on it yet.
• Model: a Qwen3-architecture decoder (8 layers, width 832, 13 heads of 64), byte-level BPE with 4,096 tokens and one token per digit, trained from scratch on 3B tokens (packed 1,024-token sequences) with Muon for the weight matrices and AdamW for the embeddings.

Results

Trigger accuracy: for held-out story pairs (a story and its spooky twin) under headers at minutes never seen in training, how often the model finds the right version more likely, balanced between night and day. Spooky rates: 200 stories sampled at held-out night and day minutes, labelled by a naive-Bayes word classifier (97% of spooky twins and 1% of normal stories are called spooky).

Files

• model.safetensors: weights for transformers (Qwen3ForCausalLM, fp32).
• onnx/model.onnx and onnx/model_quantized.onnx: ONNX with KV cache, fp32 and 8-bit (74.1 MB, what the demo loads).

Limitations

A small research model: it writes simple English children's stories and little else. The spooky stories are mild and formulaic (a child notices a strange pale shape, watches it, and it drifts away). The time trigger is a harmless stand-in for a backdoor, built to study how small a model can learn one.