TinyTurk v2.7d — BigStories

TinyTurk v2.7d BigStories is a 1.12M-parameter Turkish causal language model, trained from scratch on a 49,000-story subset of the Turkish TinyStories dataset. It produces fluent, grammatically correct Turkish sentences but does not maintain long-range narrative coherence due to the 128-token context window.

Model Summary

Training

Trained on umarigan/tinystories_tr (Turkish translation of TinyStories):

• 49,000 training stories (subset of 1.8M)
• 500 validation stories
• 500 test stories

Training time: ~26 minutes on a single modern GPU.

Evaluation

Language Modeling

Turkish Structural Diagnostics

Same diagnostic suite as v2.7d:

Generation Samples

Prompt: "Bir zamanlar kucuk bir kedi vardi."

Bir zamanlar kucuk bir kedi vardi. Kedi henuz uc yasindaymis ve uzgun hissediyordu. Kedi itfaiye aracini geride kaldi ve birlikte ahira dogru yola ciktilar. Bir gun kedi ve kiz o kadar cok mutlu oldular ki...

Observation: Sentences are grammatically correct Turkish. However, narrative coherence across sentences is limited.

Usage

Install:

pip install torch sentencepiece huggingface_hub

Python:

import torch
import sentencepiece as spm
from huggingface_hub import hf_hub_download
from model import TinyTurkGPTV27, TinyTurkV27Config

REPO = "stunmuffin/TinyTurk-v2.7d-bigstories"

ckpt_path = hf_hub_download(REPO, "pytorch_model.bin")
tok_path = hf_hub_download(REPO, "tokenizer_v11_bpe_512.model")

ckpt = torch.load(ckpt_path, map_location="cpu")
config = TinyTurkV27Config(**ckpt["config"])
model = TinyTurkGPTV27(config)
model.load_state_dict(ckpt["model_state_dict"])
model.eval()

sp = spm.SentencePieceProcessor(model_file=tok_path)

prompt = "Bir zamanlar kucuk bir kedi vardi."
ids = [sp.bos_id()] + sp.encode(prompt, out_type=int)
input_ids = torch.tensor([ids])

with torch.no_grad():
    out = model.generate(input_ids, max_new_tokens=100,
                         temperature=0.9, top_k=40)

print(sp.decode(out[0].tolist()))

See example.py for a runnable script.

Limitations

Important: This model has several known limitations:

1. Narrative coherence: Due to the 128-token context window, the model cannot maintain a consistent story across more than a few sentences.
2. Translated data: The training corpus is an automatic translation of TinyStories (English). Some English proper names (Jack, Alice, Tim) may appear in Turkish output. Some phrases are literal translations.
3. Small vocabulary: 512 BPE subwords. Rare Turkish words may be split awkwardly.
4. Short context: 128 tokens. Not suitable for long-form generation.
5. No instruction tuning: Raw pretrained language model.
6. No safety alignment: Outputs may be incoherent or inappropriate.

Intended Use

• Research on small-scale language modeling
• Turkish sentence-level generation experiments
• Educational demonstrations of transformer internals
• On-device inference experiments

Out-of-Scope Use

• Long-form storytelling
• Instruction following / chat
• Any safety-critical application

Comparison with TinyTurk-v2.7d

The original v2.7d excels at structural grammatical tasks. BigStories excels at sentence-level fluency. Neither maintains long-form narrative coherence.

Citation

@misc{tinyturk_v27d_bigstories_2026,
  title = {TinyTurk v2.7d BigStories: Turkish 1.12M-parameter LM},
  author = {TinyTurk Project},
  year = {2026}
}

TinyTurk v2.7d BigStories is an experimental research artifact. Not a product.