ThoxLLM-327M-v2
From-Scratch THOX Language Model — a THOX language model trained from scratch: a 327M-parameter decoder with a standard SwiGLU feed-forward design, built for clean GGUF export
Safetensors weights
Your AI. Your Data. Your Rules.™
Status: PUBLISHED — RESEARCH CHECKPOINT. A THOX language model trained from scratch (327M-parameter decoder, custom thoxllm architecture). No THOX GATE49 result is published for this model. See Files for exactly what this repo contains.
Summary
ThoxLLM-327M-v2 is a THOX language model trained from scratch: a 327M-parameter decoder with a standard SwiGLU feed-forward design, built for clean GGUF export. It is fully THOX-owned. For llama.cpp and Ollama use, the GGUF builds live in Thox-ai/ThoxLLM-327M-v2-GGUF.
Your AI. Your Data. Your Rules.™
Highlights
- ThoxLLM-327M-v2 is a THOX language model trained from scratch: a 327M-parameter decoder with a standard SwiGLU feed-forward design, built for clean GGUF export.
- This checkpoint uses the custom THOX
thoxllmdecoder; itsconfig.jsoncarries noarchitectureskey, so vanillaAutoModelloading may not work.
Specifications
| Base model | See card body |
| Parameters | 327M |
| Formats | Safetensors weights |
| License | Apache-2.0. Trained from scratch by THOX.ai LLC; the model and additions in this repo are covered under Apache-2.0. |
| This repository contains | Full weights and tokenizer in this repo |
Files
| File | Size | SHA-256 |
|---|---|---|
model.safetensors |
648,339,368 B | 91c71f54973df679c673e9769a3be84a2f1ad7872ccd1aca41144bcc3e879066 |
tokenizer.json / loss-curve.svg / .gitattributes / eval-curve.tsv / config.json / tokenizer_config.json |
— | Supporting files (see repo) |
Architecture
| Field | Value |
|---|---|
| Architecture | Custom THOX decoder (standard SwiGLU) |
| Layers | 20 |
| Hidden size | 1024 |
| Attention heads | 16 |
| KV heads | 8 (GQA) |
| FFN / intermediate | 3413 |
| Vocab | 50,304 |
| Max context | 2048 |
| Tied embeddings | yes |
| dtype | bfloat16 |
Run it
This checkpoint uses the custom THOX thoxllm decoder; its config.json carries no architectures key, so vanilla AutoModel loading may not work. Load it with the THOX thoxllm package, or use the GGUF build:
hf download Thox-ai/ThoxLLM-327M-v2-GGUF --include "*.gguf" --local-dir ./ThoxLLM-327M-v2-GGUF
ollama run hf.co/Thox-ai/ThoxLLM-327M-v2-GGUF:Q4_K_M
Platforms
| Platform / device | Run path |
|---|---|
| Hugging Face | Download the builds listed under Files from this repo (see Run it) |
Intended use
- On-device and edge text generation within the THOX stack.
- Not a safety-aligned public assistant unless deployed behind THOX guardrails.
- Out of scope: medical, legal, financial, safety-critical, or other high-stakes decisions without human review; unattended control of physical devices without a confirmation step.
Training data and procedure
Trained by THOX from scratch as part of the THOX small-model research line. The corpus and procedure are as described in this card at release; no external base model was used.
Evaluation
No THOX GATE49 result is published for this model.
Limitations
- A 327M base model: completion-style behaviour, limited reasoning and factual reliability; verify important answers.
- Behaviour and bias reflect the training data, which is mostly English; quality in other languages is lower.
- Custom architecture: standard loaders may not load the safetensors checkpoint directly (see Run it).
License
Apache-2.0. Trained from scratch by THOX.ai LLC; the model and additions in this repo are covered under Apache-2.0.
THOX.ai — Your AI. Your Data. Your Rules.™
Provenance
Published by THOX.ai as Thox-ai/ThoxLLM-327M-v2. THOX from-scratch research line; no upstream base model. This card was conformed to THOX Model Card Standard v1.0 on 2026-10-31 (frontmatter, tags, header, files, and section structure); the technical content is as published at release.
Citation
@misc{thoxai2026thoxllm327mv2,
title={ThoxLLM-327M-v2},
author={THOX.ai},
year={2026},
url={https://huggingface.co/Thox-ai/ThoxLLM-327M-v2}
}