NeoHorse-1-9B-Uncensored GSQ-RCO GGUF
Mixed-precision GSQ-RCO GGUF quantizations of TokenRhythm/NeoHorse-1-9B, optimized for efficient, high-fidelity local inference in llama.cpp.
The model supports a context length of up to 262,144 tokens (262K).
Why GSQ-RCO?
Standard quantization severely degrades hybrid SSM-Transformer models (e.g. stock low-bit quantizations cause recurrent state drift and degradation in reasoning coherence).
GSQ-RCO solves this through two targeted techniques:
- RCO: Keeps all 48 critical SSM parameters in unquantized BF16.
- GSQ: Optimizes per-layer precision guided by Hessian sensitivity calibration.
GGUF Files
| File | bpw | Size |
|---|---|---|
| NeoHorse-1-9B-GSQ-RCO-IQ2_XS.gguf | 2.80 | 3.13 GB |
| NeoHorse-1-9B-GSQ-RCO-IQ3_XXS.gguf | 3.00 | 3.36 GB |
| NeoHorse-1-9B-GSQ-RCO-IQ3_S.gguf | 3.50 | 3.92 GB |
| NeoHorse-1-9B-GSQ-RCO-IQ3_M.gguf (Recommended) | 3.75 | 4.20 GB |
| NeoHorse-1-9B-GSQ-RCO-IQ4_XS.gguf | 4.25 | 4.76 GB |
Usage
llama.cpp
Interactive Chat
llama-cli \
-m NeoHorse-1-9B-GSQ-RCO-IQ3_M.gguf \
-ngl 99 \
-fa \
-b 2048 \
-c 16384 \
-cnv
Local Server (OpenAI-Compatible)
llama-server \
-m NeoHorse-1-9B-GSQ-RCO-IQ3_M.gguf \
-ngl 99 \
-fa \
-b 2048 \
-c 32768 \
--port 8080
128k Long-Context (Low-VRAM)
Using 4-bit KV cache (-ctk q4_0 -ctv q4_0) allows 131k context to run within ~6 GB VRAM on consumer 8GB GPUs:
llama-server \
-m NeoHorse-1-9B-GSQ-RCO-IQ3_M.gguf \
-ngl 99 \
-fa \
-b 2048 \
-c 131072 \
-ctk q4_0 -ctv q4_0 \
--port 8080
Ollama
ollama run hf.co/abenzerps/NeoHorse-1-9B-Uncensored-GSQ-RCO-GGUF:IQ3_M
Attribution & License
- Distributed under the Apache-2.0 License, matching upstream TokenRhythm/NeoHorse-1-9B.
- Reference GGUF release: TokenRhythm/NeoHorse-1-9B-GGUF.