thox-micro-125m-instruct-GGUF

GGUF Builds of the Device-Role Responder — GGUF builds of Thox-ai/thox-micro-125m-instruct

GGUF: Q4_0, Q4_K_M, thox-micro-125m-instruct.f16.gguf

Your AI. Your Data. Your Rules.™

Status: PUBLISHED — GGUF BUILDS. GGUF builds of Thox-ai/thox-micro-125m-instruct, the instruction-tuned device-role responder. The model card of record is the main repository's. See Files for exactly what this repo contains.

Summary

GGUF builds of Thox-ai/thox-micro-125m-instruct. Read that card for what the model is and is not: a THOX device-role responder, not a general assistant.

Q4_0 is the device target: the FFN dimension (2048) divides cleanly into Q4_0 blocks. The files are the same size as the untuned base because the ChatML tokens reuse unused vocabulary rows. Embedded metadata: tokenizer.ggml.eos_token_id = 50258 (<|im_end|>), padding_token_id = 50258, and the ChatML tokenizer.chat_template; both <|endoftext|> and <|im_end|> are end-of-generation tokens.

Highlights

• GGUF builds of Thox-ai/thox-micro-125m-instruct.

Specifications

Files

Run it

llama-completion -m thox-micro-125m-instruct.Q4_0.gguf -no-cnv --temp 0 -n 96 -p $'<|im_start|>system\nYou are a THOX model running locally on THOX hardware. Your AI. Your Data. Your Rules.<|im_end|>\n<|im_start|>user\nIs my data leaving this device?<|im_end|>\n<|im_start|>assistant\n'

Or with the embedded template: llama-server -m thox-micro-125m-instruct.Q4_0.gguf --jinja.

ollama run hf.co/Thox-ai/thox-micro-125m-instruct-GGUF:Q4_K_M

Platforms

Intended use

• On-device THOX identity, status, and data-locality responses, with general requests routed to a larger model.
• Out of scope: general chat, factual QA, arithmetic, code, and any safety-critical use.

Training data and procedure

Trained by THOX from scratch as part of the THOX small-model research line. The corpus and procedure are as described in this card at release; no external base model was used.

Evaluation

Measured on a host machine (not target device hardware): Q4_0 at 191 MB peak RSS, ~450 tok/s decode, 7 of 7 clean stops with its system prompt. Not yet validated on target device hardware. No THOX GATE49 result is published for this model.

Limitations

• A device-role responder, not a general assistant; see the source card for the known system-prompt conditioning issue.
• Quantization trades accuracy for size.

License

Apache-2.0. Copyright (c) 2026 THOX.ai LLC. Quantized from the THOX instruct build; covered under Apache-2.0.

THOX.ai — Your AI. Your Data. Your Rules.™

Provenance

Published by THOX.ai as Thox-ai/thox-micro-125m-instruct-GGUF. THOX from-scratch research line; no upstream base model. This card was conformed to THOX Model Card Standard v1.0 on 2026-10-31 (frontmatter, tags, header, files, and section structure); the technical content is as published at release.

Citation

@misc{thoxai2026thoxmicro125minstructgguf,
      title={thox-micro-125m-instruct-GGUF},
      author={THOX.ai},
      year={2026},
      url={https://huggingface.co/Thox-ai/thox-micro-125m-instruct-GGUF}
}