TinyStories Llama2 1M (tiny1m) GGUF & HF Validation Suite

This repository provides ultra-lightweight Llama2 model files across various formats (both GGUF and Hugging Face / Safetensors), trained on the TinyStories dataset and optimized for compatibility with Andrej Karpathy's llama2.c and llama.cpp.

Why this repository exists

When developing a custom LLM inference engine, debugging with a full-sized model is slow. This suite offers a true 1M parameter scale model (~1MB to ~4MB depending on the quantization format), allowing developers to validate their loaders, serialization, quantization kernels, and inference logic step-by-step with maximum efficiency.

---

📂 Repository Structure & File Descriptions

1. GGUF Formats (Root Directory ./)

A comprehensive validation suite converted for llama.cpp and compatible engines. Every compiled quantization variant available in the root directory is explicitly covered below:

2. Llama2.c & Base Tokenizer Assets (Root Directory ./)

Files optimized for execution within the native llama2.c ecosystem:

• model.bin: A single flat binary file containing all network weights, custom layout arrays, and pre-computed RoPE frequencies structured specifically for run.c.
• tokenizer.bin: The structural binary version of the 512-vocab tokenizer compiled for rapid streaming and direct parsing by run.c.
• tokenizer.model: The master SentencePiece tokenizer model file (512 vocabulary size, identical to the stories260k standard) kept at the root for upstream conversion tools and local reference.

3. Hugging Face Native Format (./hf/)

This directory contains the standard files required to load the model using the PyTorch transformers library:

• hf/model.safetensors: The raw, unquantized model weights stored securely in Safetensors format.
• hf/config.json: The architectural configuration file defining hyperparameters (layers, heads, dimensions).
• hf/generation_config.json: Default parameters optimized for text generation (temperature, top_p, etc.).
• hf/tokenizer.model: A redundant copy of the 512-vocab SentencePiece tokenizer model placed inside the directory for seamless Hugging Face API resolution.

---

🚀 Quick Start & Usage Examples

A. Running GGUF via llama.cpp

To verify your local setup or compare tokens using the official native utilities:

./llama-cli -m tiny1m.Q4_K_M.gguf -p "Tom and Jerry are " -n 64 --temp 0.0

B. Running via llama2.c (Native Binary)

The model.bin is fully compatible with the 512-vocab tokenizer.bin derived from the stories260k asset pipeline.

./run model.bin -z tokenizer.bin -i "Tom and Jerry are " -n 64

C. Loading Hugging Face Formats via Python

You can import the Hugging Face variant directly into Python using the transformers library.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo_id = "shibatch/tiny1m"

# The library automatically looks into the hf/ folder using the subfolder argument
tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="hf")
model = AutoModelForCausalLM.from_pretrained(repo_id, subfolder="hf")

prompt = "Tom and Jerry are "
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(
        **inputs, 
        max_new_tokens=64, 
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

📝 Model Specifications

The network architecture features an unshared output layer (lm_head) to keep memory structures consistent with standard Llama 2 definitions. Thanks to the highly optimized 512 vocabulary size, the token embedding and output layers remain extremely lightweight.

• Architecture: Llama 2 (Scaled-down variant)
• Dataset: TinyStories
• Total Parameters: ~1M (Exactly 896,256 parameters)
• Vocabulary Size: 512 (Uses the stories260k compatible 512-vocab tokenizer layout)
• Hidden Size (hidden_size): 128
• Number of Hidden Layers (num_hidden_layers): 4
• Number of Attention Heads (num_heads): 2
• Number of Key-Value Heads (num_kv_heads): 2
• Intermediate Size (intermediate_size): 352
• Max Position Embeddings (max_position_embeddings): 256

📜 Acknowledgments & License

• Original Implementation: Inspired by Andrej Karpathy's llama2.c project.
• Dataset: TinyStories dataset.
• License: MIT License. You are free to use, modify, and distribute these assets for any purpose.