Newer version: quantumaikr/max-0.1b-v1 continues this model to 10.0B training tokens (better validation loss and public Korean task scores). / 후속 모델: max-0.1b-v1(학습 100억 토큰).
A small Korean-first base language model (109.5M parameters) for industrial small functions such as document classification, request routing, field extraction and form filling. This is an early base model (not instruction-tuned). It is intended as a starting point for task fine-tuning and for research, not for direct end-user use.
한국어 중심의 소형 기초 언어모델(1.095억 매개변수)입니다. 문서 분류, 요청 분기, 항목 추출, 양식 작성 같은 산업용 작은 기능의 출발점으로 쓰기 위한 초기 기초 모델이며, 지시 따르기 미세조정 전 단계입니다.
| File | Use |
|---|---|
model.safetensors, config.json, tokenizer.json |
Transformers (Qwen3ForCausalLM-compatible layout) |
gguf/max01b-v0-f16.gguf (221 MB) |
llama.cpp, full precision |
gguf/max01b-v0-q8_0.gguf (118 MB) |
llama.cpp, 8-bit |
gguf/max01b-v0-q4_k_m.gguf (73 MB) |
llama.cpp, 4-bit, runs on ordinary CPUs |
Context length: 1,024 tokens. Vocabulary: 32,000 (Korean-optimized, digits split). Input text should be Unicode NFC-normalized.
from transformers import AutoModelForCausalLM, PreTrainedTokenizerFast
tok = PreTrainedTokenizerFast.from_pretrained("quantumaikr/max-0.1b-v0")
model = AutoModelForCausalLM.from_pretrained("quantumaikr/max-0.1b-v0")
llama-cli -m max01b-v0-q4_k_m.gguf -p "오늘 회의에서 논의된 안건은" -n 64
Validation (bits per byte, lower is better): Korean web 0.90 · Korean documents 0.83 · English 1.03.
| Task (0-shot, n=300) | Accuracy | Chance |
|---|---|---|
| ARC-Easy | 0.43 | 0.25 |
| PIQA | 0.60 | 0.50 |
| HellaSwag | 0.29 | 0.25 |
| KoBEST COPA | 0.56 | 0.50 |
| KoBEST HellaSwag | 0.32 (norm 0.41) | 0.25 |
| KoBEST BoolQ | 0.49 | 0.50 |
Few-shot industrial tasks (n=40 each, before fine-tuning): document-type classification 0.63 (calibrated), complaint classification 0.45, request routing 0.28, JSON output parse rate 0.48–0.58. Field extraction and number-faithful summarization are not usable without fine-tuning.
All training data is publicly released under the Open Data Commons Attribution License (ODC-By 1.0):
These datasets are derived from Common Crawl and are also subject to the Common Crawl Terms of Use.
Apache License 2.0.
PyTorch training checkpoints for continued pretraining / fine-tuning (moved here from GitHub releases):
| file | what |
|---|---|
checkpoints/max01b-v0-stable_pre_decay_2.pt |
step 4818, 2.40B tokens, full LR, includes optimizer state (resume pretraining) |
checkpoints/max01b-v0-final.pt |
step 5595, 2.81B tokens after cooldown (eval / fine-tune) |
Verify with checkpoints/SHA256SUMS (sha256sum -c SHA256SUMS).