Nietzsche GPT (15M params)

A small (~15M parameter) GPT-2-style decoder-only Transformer trained from scratch on public-domain English translations of Friedrich Nietzsche's major works (via Project Gutenberg), as a first from-scratch LLM training exercise.

Training details

• Architecture: 6 layers, 6 heads, 384 embedding dim, 256 token context
• Tokenizer: custom byte-level BPE, 8,000 vocab, trained on the same corpus
• Trained on a single Colab T4 GPU with mixed precision (fp16)
• Corpus: ~10 Nietzsche works (Thus Spake Zarathustra, Beyond Good and Evil, etc.)

Limitations

This is a small model trained on a small corpus for a small number of epochs. It captures some of Nietzsche's vocabulary and sentence rhythm but should not be relied on for factual or coherent long-form output. Built as a learning exercise, not a production model.