NeoHorse-1-4B GGUF

Community GGUF quantizations of TokenRhythm/NeoHorse-1-4B.

☕ If this GGUF made your day easier, a coffee would make mine.
Send a coffee ☕
I build and test these releases myself. Your coffee helps keep me going.
Thank you for supporting this work.

About NeoHorse-1-4B

NeoHorse-1-4B is an approximately 4B-parameter text-only causal language model from TokenRhythm, post-trained from Qwen3.5-4B for text-based agent harnesses, tool use, coding, instruction following, and conversational use. The upstream card reports a native context length of 262,144 tokens; practical context depends on runtime, memory, KV-cache settings, and workload.

See the official upstream model card for the original benchmark table, deployment notes, technical report, and model details. Vision weights are not included in the upstream text-only release.

These releases change the storage format through GGUF quantization only. No training or fine-tuning was performed here.

Fidelity measurements

The table covers every one of the 26 published GGUF files. Size (GB) uses decimal gigabytes (1 GB = 1,000,000,000 bytes). It averages eight chunks from each of wiki.test.raw and wiki.valid.raw, using a 4,096-token context and the same llama.cpp runtime against the BF16 GGUF reference. The average BF16 PPL in this evaluation was 8.635307. Lower Mean KLD, ΔPPL, and RMS Δp, and higher Top-1 agreement, indicate closer next-token behavior to BF16. These are fidelity measurements, not a direct percentage of capabilities retained.

Sweet spots: choose Q8_0 or Q6_K for fidelity, Q5_K_M for the balanced default, and Q4_K_M for a smaller practical package.

The fixed hold-out corpus is disjoint from the calibration inputs. See the ladder quality summary, supplemental quality summary, and the reproducibility manifest for the exact split, hashes, and runtime profile.

Behavior and runtime validation

BF16 baseline probes covered English, Vietnamese, Chinese, coding, plain JSON, and the upstream-style tool-call text format. Those probes generated usable responses for the tested prompts. The strict llama.cpp --json-schema probe failed during runtime sampler initialization and is recorded as a runtime compatibility limitation, not counted as a model capability result. The tool-call probe tested serialization format only; it was not an end-to-end tool executor test.

BF16 and every published quantization loaded and generated non-empty output on one NVIDIA A10M using the same GPU-0 profile. Low-bit files can pass this compatibility gate while still producing poor responses; the fidelity table is why Q3 and lower are marked experimental.

Quick start

For the balanced starting point:

./llama-cli \
  -m NeoHorse-1-4B-Q5_K_M.gguf \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

The GGUF contains the converted chat template. Reduce -ngl or the context size if the available VRAM is smaller.

Reproducibility and validation

The locked upstream revision is 56f0584bb40578a2c33b1b40a08ccd17243ad710. The BF16 source GGUF SHA256 is 10aae5451e1c0b6089b25f68cfe32cd32a4bdcb92e30008e083e6ba351f3cbd8; the NeoHorse-specific combined imatrix SHA256 is 8c66b45b73826b0b00fdd973d308d8f81f7db0f8a3b97cf93bc7813d722d6515.

The package includes SHA256SUMS.txt, the quantization manifest, source and calibration manifests, the runtime version, and the copied upstream chat template. Detailed conversion, quantization, smoke, behavior, and fidelity logs remain local under reports/neohorse-1-4b/ and are not uploaded.

License and attribution

NeoHorse-1-4B is released upstream under the Apache License 2.0. The upstream license and attribution notices are included in LICENSE.

These are community GGUF quantizations, not an official TokenRhythm release or endorsement.

Supplemental variants

The nine-file ladder above is complete. These additional llama.cpp formats are built directly from the same locked NeoHorse-1-4B BF16 source and model-specific imatrix. A row is marked published only after runtime smoke, hold-out fidelity, SHA256, and Hugging Face metadata verification pass. IQ1_S, TQ2_0, and TQ1_0 are published for compatibility/experimentation only and are not recommended for normal use because their fidelity drift is severe. Q6_K_F16 is a hybrid variant: quantized tensors use Q6_K while token embeddings remain F16, so it is larger than standard Q6_K. A failed candidate would remain in the status ledger but be withheld.

See the supplemental status ledger and supplemental fidelity summary.