FahQgpt1.1 nano
由 Bilibili @一片烂海苔 自己训练的 **145,910,912 参数(约146M)**实验性英文对话模型。使用单张RTX5060Ti16GB;从自训练FahQgpt1.0 Nano底座继续训练,未使用第三方预训练权重。
A tiny experimental English conversational model continued from the author’s own from-scratch base. The included web interface runs the model directly: no retrieval, calculator, canned-answer lookup, external model API or internet access during inference. Public synthetic dialogue text was used during training.
Download and run / 下载运行
Python3.10+。先安装适合自己CPU/CUDA环境的PyTorch,然后:
pip install huggingface_hub
hf download txktxkabcd/fahqgpt-1.1-nano --local-dir fahqgpt-1.1-nano
cd fahqgpt-1.1-nano
pip install -r requirements.txt
python -m fahqgpt.chat_validation --prompt "who are you"
python -m fahqgpt.chat_validation --device cpu --prompt "hello"
python -m fahqgpt.serve_11 --port 7861
浏览器打开 http://127.0.0.1:7861 。上下文默认关闭,多轮仍不稳定。CLI不带--prompt进入交互模式。CUDA可用时自动使用CUDA,否则使用CPU;CPU速度较慢,数值差异可能影响输出。
The PyTorch package uses safetensors and the supplied entrypoints; it does not implement Transformers AutoModel. A separately converted F16 GGUF is also available below. No original training workspace or .pt file is needed.
F16 GGUF / 给只支持GGUF的软件
Download FahQgpt1.1-nano-F16.gguf (~293MB /279.4MiB). Import the GGUF into a recent llama.cpp-compatible application and use the embedded chat template. The model is small enough to keep F16; no Q4 conversion is supplied. FP32-to-F16 rounding still applies.
llama-server -m FahQgpt1.1-nano-F16.gguf -c 1024 --jinja --port 8080
Open http://127.0.0.1:8080 . Start with temperature0, repeat penalty1.12, maximum160new tokens and short English prompts. Set context to1024. Do not select ChatML/Llama3 or add a BOS token: this model uses plain User:/Assistant: labels. If your app overrides the embedded template, use this exact one-turn prompt, ending immediately after the colon:
User: hello
Assistant:
Stop token is <|eot|>(ID4); <|endoftext|>(ID0) is also an end marker. Avoid adding a default system prompt. Multi-turn remains experimental; start a fresh conversation when responses drift. The embedded template intentionally omits system messages because this chat run did not train a system role.
Verified on llama.cpp build9873(a4107133a), CPU: four tokenization probes match the original tokenizer, the embedded template matches the original prompt, and hello/identity/story prompts generate nonempty English. See evaluation/gguf_checks.json. The application’s sampling/repetition behavior may differ from the supplied PyTorch runtime, so outputs are not guaranteed identical. Compatibility with every third-party GGUF app is not established.
Architecture
| Item | Value |
|---|---|
| Parameters | 145,910,912 (unique/tied counted once) |
| Layers / hidden size | 14 / 896 |
| Query / KV heads | 14 / 2 (GQA), head dimension64 |
| FFN hidden | 2432, SwiGLU |
| Vocabulary | 32,002 |
| Context window | 1,024 tokens including prompt and response |
| Position / normalization | RoPE(theta10,000), RMSNorm |
| Embeddings | Shared input/output |
| Storage / compute | FP32 safetensors, CUDA bf16 autocast or CPU fp32 |
The safetensors file stores emb.weight once; the supplied loader restores the lm_head.weight alias. All128 saved tensors were checked exactly against the source checkpoint after export.
Training provenance
- Author’s own base run: 14,495steps and1,899,888,640tokens. The published1.0 repository is https://huggingface.co/txktxkabcd/fahqgpt-1.0-nano ; this1.1 run starts from the original local base checkpoint, not a dequantized public GGUF.
- This chat run:36,286 training conversations and543 validation conversations. Training pool contains8,246,602 inputtokens and5,549,915 assistant-supervised tokens, including11,233 multi-turn conversations.
- Public English dialogue text from HuggingFaceTB/smol-smoltalk and the everyday subset of HuggingFaceTB/smoltalk, plus locally authored examples. Sources include synthetic text; no third-party model weights or inference service were used for this run.
- Source snapshots:smol-smoltalk@f73fe857d519ff6ac5af2ea67c4d3834da7b8bcc and smoltalk@5feaf2fd3ffca7c237fc38d1861bc30365d48ffa. Downloaded raw shards were SHA256-verified.
- Filtered for short English conversations, tokenlimit512; removed many assigned personas, task labels and tiny answers. Filtering is imperfect. Group split by normalized first-user prompt; no shared split group.
- 3,200steps, microbatch4, accumulation4, effective batch16; AdamW, peakLR1e-4,100warmupsteps, cosine decay. Length-bucket sampling with replacement:51,200main conversation draws; this does not mean every training row was visited.
- A24-example locally authored identity supplement is sampled in1%of microbatches. About27minutes including checkpoint/evaluation overhead on oneRTX5060Ti16GB; peak allocated trainingVRAM4.59GiB.
- Token-weighted validation NLL on the first160held-out examples:2.5112 before to1.5761 after. This is not an accuracy score or an independent broad benchmark.
Template and decoding
User: {user}
Assistant: {answer}<|eot|>
Use the supplied tokenizer/prompt builder: prompt and assistant completion are tokenized separately with a leading space on assistant text, matching training. Web default:160new tokens, deterministic greedy decoding with generated-token repetition penalty1.12 and a repeated-four-token-phrase ban. Stops at<|eot|>or<|endoftext|>. These controls reduce loops; they do not supply answers.
Actual examples and limitations
- who are you → “I’m FahQgpt1.1 nano, a small language model trained from scratch.”
- what can you do → “I can help with writing, questions, ideas, and casual conversation.”
- write a story → generates a city-at-dusk paragraph; it can have weak plot, repeated imagery and invented spelling.
This is a learning/fun experiment. It often misreads requests, gives false facts, invents words and derails with conversation history. Misspelled praise such as “wow,you are godd” still receives a generic greeting. Capability descriptions are generated text and are not proof of reliable abilities. English is the trained chat focus; Chinese chat is not established.
See evaluation/ for unedited development/fresh-prompt generations, actual local HTTP checks, and data-cleanup statistics. These prompts were used for model review; they are not a blind benchmark. The full corpus, optimizer state, resumable checkpoints, and credentials are not included.
License
Apache-2.0, following the author’s previous release. Dataset provenance and source terms remain relevant; no claim of complete manual audit of public training text.
Files
FahQgpt1.1-nano-F16.gguf: standalone F16 llama.cpp-compatible model (~293MB), with embedded tokenizer and chat template. model.safetensors: exact FP32 inference weights (~584MB). tokenizer.json: original tokenizer. config.json: architecture fields used by the custom loader. fahqgpt/: runnable inference and local web UI. training_provenance.json and evaluation/: reproducibility/review records. SHA256SUMS.json: release file checksums.