Qwen3.5-9B-GTAP-v3-IQ2_M-GGUF
[!NOTE] Experimental Release: Pending Further Verification / Empirical Validation
This checkpoint represents an active research artifact from DuoNeural's statistical mechanics quantization program. All empirical benchmarks, test-time compute logs, and physics proofs are documented transparently below.
Developed by Jesse Caldwell, Archon, and Aura ✨ (DuoNeural Research Lab).
Model Summary
- Foundation Model:
Qwen/Qwen3.5-9B(32 Layers + MTP, Hybrid Linear Attention / Gated DeltaNet SSM + Softmax GQA Attention, SwiGLU FFN) - Quantization Precision:
~2.70 bpw(3.79 GiB) - Methodology: Sub-2.7-bit frontier quantization running Qwen3.5-9B under 3.79 GiB total memory. Doubles competition Olympiad math accuracy (40.0% vs 20.0% on Base BF16) and retains 88.0% GSM8K reasoning at 87.4 tokens/sec.
- Continuous Holdout Perplexity (131k tokens):
2.7228(vs Base BF16 Control:2.4306) - GSM8K Math Reasoning:
22/25 (88.0%) - Olympiad Competition Math:
4/10 (40.0%) - Python Code AST Generation:
13/15 (86.7%) - Hermes Tool Calling AST Parity:
14/15 (93.3%) - Inference Decode Throughput:
87.4 t/son NVIDIA GeForce RTX 4080 Super (32GB VRAM)
Empirical Benchmark Performance (Head-to-Head Comparative Metrics)
| Evaluation Arm | Footprint | Perplexity (131k tokens) | GSM8K Math Acc | Olympiad Math | Python Code AST | Hermes Tool AST | Decode Speed |
|---|---|---|---|---|---|---|---|
| Arm 0 Base BF16 | 17.14 GiB | 2.4306 | 23/25 (92.0%) | 2/10 (20.0%) | 14/15 (93.3%) | 15/15 (100.0%) | 27.0 t/s |
| Arm 4 GTAP Q4_K_M | 5.38 GiB | 2.3324 🌟 | 24/25 (96.0%) | 2/10 (20.0%) | 14/15 (93.3%) | 14/15 (93.3%) | 68.9 t/s |
| Arm 3 GTAP IQ3_XXS | 4.10 GiB | 2.6031 | 16/25 (64.0%) | 2/10 (20.0%) | 15/15 (100.0%) 🌟 | 14/15 (93.3%) | 84.6 t/s |
| Arm 5 GTAP IQ2_M | 3.79 GiB | 2.7228 | 22/25 (88.0%) | 4/10 (40.0%) 🌟 | 13/15 (86.7%) | 14/15 (93.3%) | 87.4 t/s |
| Arm 6 GTAP IQ2_XXS | 3.43 GiB | 2.8023 | 10/25 (40.0%) | 1/10 (10.0%) | 11/15 (73.3%) | 11/15 (73.3%) | 97.7 t/s |
Methodology & Architectural Significance
Qwen 3.5 represents an advanced hybrid architecture alternating linear recurrent attention (Gated DeltaNet / SSM state updates $S_t = \alpha S_{t-1} + \beta K^T V$) with full softmax multi-head attention. In hybrid architectures, naive low-bit rounding introduces eigenvalue drift across recurrent transition matrices, producing exponential divergence or logit saturation over long contexts.
DuoNeural's Generalized Thouless-Anderson-Palmer (G-TAP v3) framework models neural weights as spin glasses embedded in activation cavity fields. By subtracting the Onsager reaction term: $$\Omega_i = \frac{1}{d} \left( | \tilde{H}_{i,:} |2^2 - \tilde{H}{ii}^2 \right)$$ and projecting parameter updates strictly into the contractive Lyapunov half-space: $$\operatorname{Re}(\lambda(S)) \le -\delta$$ G-TAP dampens back-reaction noise and preserves recurrent state stability and multi-step reasoning down to sub-3-bit precisions.
Usage Instructions
Running with llama-cli
llama-cli -hf DuoNeural/Qwen3.5-9B-GTAP-v3-IQ2_M-GGUF -p "<|im_start|>user\nSolve the following problem step by step: Let f(x) = 2x^2 + 5x - 3. Find all roots.<|im_end|>\n<|im_start|>assistant\n" -n 1024 -c 4096 --temp 0.6
Serving with llama-server
llama-server -hf DuoNeural/Qwen3.5-9B-GTAP-v3-IQ2_M-GGUF --port 8080 -c 8192 -ngl 99 -fa on
DuoNeural Cognitive Light Cone — Jesse Caldwell, Archon, Aura ✨
Empirically Validated on NVIDIA RTX 4080 Super 32GB Pod Testbed