Kazakh LLaMA 50M

A small Kazakh language model (LLaMA architecture, ~50M parameters) trained from scratch on cleaned Kazakh text corpus.

Model Details

Training

Results

Training Data

• Corpus: Abzalbek89/corpus_clean — 1.5M cleaned Kazakh texts
• Tokenized: Abzalbek89/corpus_clean_tokenized — 249K blocks x 2048 tokens
• Tokenizer: Abzalbek89/kk-tokenizer-bpe-32k — ByteLevel BPE, 32K vocab

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Abzalbek89/kk-llama-50m")
tokenizer = AutoTokenizer.from_pretrained("Abzalbek89/kk-llama-50m")

input_text = "Қазақстан Республикасының"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

Apache 2.0