BalChat — scratch research prototype
This is a small Balochi text-completion experiment initialized with random weights and a new byte-level BPE. No pretrained checkpoint or LoRA was used. The architecture uses the Llama implementation in Transformers. It is a research artifact, not a reliable conversational assistant.
Capabilities and observed limitations
Natural Balochi chat, English translation, reasoning and tool use have not been established. No native speaker approved the generated language; verified instruction pairs and native-reviewed training records are both zero. The commercial diagnostic requests produced punctuation/repetition instead of useful answers. Diagnostic outputs for this checkpoint are saved in verification.json and remain unreviewed. There is no claim of a first-ever Balochi model, billion-parameter scale, or ChatGPT/Gemini/Claude quality.
Byte encoding preserves Arabic and Latin Unicode. That does not establish fluency in either script. The commercial training pool is predominantly Latin script. Do not use these outputs as authoritative translations or facts.
Actual training run
- 5,670,912 full trainable parameters; 3,707-token vocabulary.
- Six decoder layers, width 256, MLP 768, eight query heads/four KV heads, tied embeddings, context 512. Training blocks contain at most 256 tokens.
- 16 training records / 15,818 word occurrences; 1 validation records / 1,746 words; 0 test records / 0 words.
- 90 optimizer updates over five epochs. FP32 AdamW on one existing TITAN X Pascal GPU; learning rate 0.0003, batch 1/accumulation 8, seed 42.
- Validation NLL 8.258177 → 5.727272; perplexity 3859.053010 → 307.130297.
- The tokenizer saw only training documents. Frozen document/source partitions were checked; validation selected the checkpoint. Test data, where present, was evaluated after selection.
Repeated epochs are repeated exposure to the same text, not newly collected words. The research training pool includes the commercial text; these are not independent data collections. Perplexity depends on the tokenizer and cannot be compared directly across the two runs or with Qwen. Loss reduction does not establish conversational quality.
Artifact verification
Original GPU-reported file checksums were checked after retrieval. Saved model tensors are finite, have the selected state digest, and reproduce validation loss on CPU. Tokenizer input-field metadata was repaired without changing the learned tokenizer.json bytes or model weights. See release-metadata.json and verification.json for the released hashes and diagnostics. Historical training.json uses an imprecise timing label: the soft elapsed loop budget includes inter-epoch evaluation/checkpointing, excludes initial validation/final reload, and is checked before updates.
The model uses standard Transformers components and requires no custom remote code. Example libraries: transformers 4.57.6, tokenizers 0.22.2, huggingface-hub 0.36.2, safetensors 0.7.0 and an appropriate Torch build.
Terms and source code
Public weights: Apache 2.0. Training sources are CC BY 4.0; retain the source attribution in SOURCE_CREDITS.md. Code is MIT.
No raw corpus is bundled. Reproduction code, software tests and measured limitations. A much larger licensed corpus, reviewed task examples and independent native evaluation are still required for the requested assistant.