IsValorum/Thomson-1.0-Small-APEX-I-MiniPlus-V2-GGUF

Quick Navigation Index

Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.

If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.

1. Independent Benchmark of the APEX-I-MiniPlus Family (Occamy V2 Reference)
2. Bundled Model Files & Specifications
3. Comparative Quantization Analysis (vs. Flat Quants & Generic APEX)
4. Bundled Q8_0 High-Precision Multimodal Vision Projector
5. Everyday Laptop Benchmarks (DDR4 / DDR5 RAM)
6. Estimated Projections on Consumer Hardware
7. The 24GB Miracle: Full 256K Context Runs In VRAM!
8. Hardware Throughput Projections (RTX 30 / 40 / 50)
9. Recommended Generation Parameters (Thomson Reuters Official)
10. Optional Support

[!NOTE]

ARCHITECTURE SELECTION GUIDE — MINIPLUS V2 & V2.1 EDITIONS

This repository hosts the MiniPlus V2 edition of Thomson-1.0-Small. Our releases are precision-engineered for specific hardware budgets and memory topologies. V2 is NOT obsolete or "worse"; each edition serves distinct inference requirements:

• MiniPlus V2 (High Theoretical Layer Protection): On paper, V2 provides extra protective envelopes on edge layers (10 layers in IQ3_S + IQ4_NL shared experts + Q8_0 attention gates). However, in practical inference benchmarks—even across extreme long-context windows exceeding +160K tokens—there is virtually NO perceptible difference in quality or reasoning compared to V2.1.
• MiniPlus V2.1 (System RAM Streaming Specialist with Deep Context): Specially prepared to run totally or partially in system RAM (DDR4/DDR5) across massive legal and financial document contexts (up to 256k tokens). By replacing non-linear codebooks with linear Q3_K edge experts, keeping Q8_0 attention gates, and upgrading shared foundation experts to Q5_K across all 40 layers, it completely eliminates AVX2 CPU dequantization stalls (+24 to 28+ tok/s streaming). Depending on your processor and memory bandwidth (DDR4/DDR5), streaming generation in system RAM can be almost as fast as having everything in VRAM, while supporting deep context keeping the dedicated Q8_0 multimodal vision projector (mmproj) explicitly loaded in GPU VRAM for instant OCR and table parsing. It provides this massive RAM streaming acceleration for only approx. 100 MB more, which is completely negligible in system RAM.

Which one should you choose? (Official Recommendation: V2.1)

• ⭐ PRIMARY RECOMMENDATION — Thomson-1.0-Small APEX-I-MiniPlus V2.1: For virtually all users and deployments, V2.1 is the strictly recommended release. Empirically verified on WikiText-2, V2.1 achieves an outstanding Perplexity of 5.3338 ± 0.1254 (ΔPPL ≈ +0.05 from unquantized baseline (approx. 5.28)), matching the token fidelity of Q5_K / Q6_K class quantizations while weighing only approx. 14.7 GB (same footprint as Q3_K_M). Furthermore, it completely eliminates AVX2 CPU stalls, providing blistering +24 to 28+ tok/s streaming under system RAM offload.
• MiniPlus V2 Legacy: Maintained for architectural transparency and users seeking specialized configurations for their workflow.

Both editions are handcrafted and vastly outperform flat 3-bit quants and generic community APEX-I-Mini releases. To explore or download the V2.1 edition of Thomson-1.0-Small optimized for system RAM streaming, visit: IsValorum/Thomson-1.0-Small-APEX-I-MiniPlus-V2.1-GGUF

[!WARNING]

DO NOT CONFUSE APEX-I-MINIPLUS WITH GENERIC COMMUNITY APEX-I-MINI!

Regardless of release version (whether V1, V2, or V2.1), NEVER confuse handcrafted APEX-I-MiniPlus builds with generic community APEX-I-Mini releases:

• Generic Community APEX-I-Mini: Uniformly compresses all core MoE experts down to aggressive 2-bit IQ2_S (dropping below the critical quality floor), leaves the sensitive token output head unarmored at 3-bit Q3_K_M, and compresses attention projections down to Q3_K. In deep reasoning models, this triggers severe perplexity spikes, syntax errors, and broken code brackets.
• Handcrafted APEX-I-MiniPlus (All Editions by IsValorum): Every single MiniPlus release—from V1 and V2 to V2.1—is a custom tensor-by-tensor architecture that preserves uncompressed F32 router gates, armors the token output head in high-precision Q6_K, safeguards attention gates in Q8_0, and keeps core reasoning experts at or above calibrated 3-bit (IQ3_XXS/IQ3_S). Even our earlier builds vastly outperform generic community APEX recipes and flat 3-bit quants.

[!TIP]

SYSTEM RAM INFERENCE: FULL OR PARTIAL

This APEX-I-MiniPlus release is designed for full or partial system-RAM inference. Depending on the processor, memory bandwidth, and DDR4/DDR5 configuration, generation can range from 20 to 45 tok/s. With partial GPU offload, systems that cannot fit 128K or more context entirely in VRAM can place the remaining model and context load in system RAM, maintaining stable, responsive generation at longer context lengths.

---

[!IMPORTANT]

EXPLORE THE ESTABLISHED 35B MoE MINIPLUS LINEUP

These are complementary APEX-I-MiniPlus V2.1 releases, not alternate downloads of the same model. Each receives the same tensor-by-tensor approach, integrated MTP where supported, and a design suitable for full or partial system-RAM inference. Choose the model whose native strengths best fit the work you want to do:

• Qwen3.6-35B-A3B MTP APEX-I-MiniPlus-V2.1 — a versatile frontier MoE for broad reasoning, multilingual work, agents, tool use, and multimodal tasks.
  • Best for: General reasoning, agent workflows, tool calling, and flexible multimodal use.
• Qwen3.6-35B-A3B MTP APEX-I-MiniPlus-V2.1 Abliterated — the V2.1 refusal-ablated Qwen3.6 edition for users who deliberately prefer reduced refusal behavior.
  • Best for: Workflows where an abliterated Qwen3.6 variant is explicitly desired.
• Ornith 1.5 APEX-I-MiniPlus-V2.1 — a software-engineering-focused MoE designed for repository-scale coding and autonomous engineering agents.
  • Best for: Repository-scale development, multi-file code changes, and software-engineering agents.
• Tiel Coder APEX-I-MiniPlus-V2.1 — a specialist coding MoE tuned for agentic programming, iterative tool use, and implementation-heavy work.
  • Best for: Focused coding sessions, iterative debugging, and tool-driven implementation.

These remain distinct model families and editions with their own behavior and empirical results. Pick by workload and intended alignment behavior rather than treating them as interchangeable quantization variants.

---

🏅 Independent Benchmark of the APEX-I-MiniPlus Family (Occamy V2 Reference)

[!NOTE] External report: zephel01 independently benchmarked Occamy V2. The benchmark below was performed on Occamy-1.0 APEX-I-MiniPlus V2, not on this specific model. It is included as independent evidence of the broader MiniPlus quantization approach.

The APEX-I-MiniPlus quantization architecture powering this model was subjected to an extensive independent evaluation by Japanese AI researcher and evaluator zephel01 (CoolZero) on an NVIDIA RTX 5090 (32GB) workstation running llama.cpp CUDA b11027 with FlashAttention (-fa on -ctk q8_0 -ctv q8_0 -ngl 99).

The evaluation tested the APEX-I hybrid MoE engine across 348 unseeded trials on SWE-bench style multi-file Python bug-fixing tasks with hidden pytest suites (llmbench):

• L6 Multi-File Code Generation (60 tasks):
  • Context 32,768 (32K): 93.3% Resolved (46/60 tasks passed 5/5 consecutive trials; 20/20 on Easy–Hard).
  • Context 65,536 (65K): 90.0% Resolved (45/60 tasks passed 5/5 consecutive trials).
  • Match with 25–28 GB Models: Matches or exceeds the resolution rate of full 25–28 GB models (such as Ornith-1.5 and Tiel-Coder 35B-A3B) while consuming over 10 GB less VRAM (14.6 GB vs approx. 26 GB).
• Extreme Context VRAM Scaling (The Hybrid DeltaNet SSM Advantage):
  • 32K Context: 14.6 GB total VRAM allocation.
  • 65K Context: 15.1 GB total VRAM allocation (only +0.5 GB VRAM added when doubling context!).
  • Architectural Explanation: Because 30 of the 40 layers utilize Linear Attention / DeltaNet SSM ($O(1)$ constant recurrence memory), only the 10 full-attention anchor layers expand the KV cache. This proves empirically that 65,536 context runs 100% in VRAM on consumer 16GB GPUs (RTX 4080 / RTX 5080) without offloading to system RAM.
• Measured Real-World Throughput: Sustained single-stream generation of approx. 247 – 251 tok/s on NVIDIA RTX 5090.

---

Bundled Model Files & Specifications

• Base Architecture: Qwen3_5MoeForConditionalGeneration (40 layers, 256 fine-grained micro-experts with intermediate dimension 512, 8 active per token) + Vision Projector.
• Active Parameters: approx. 3.2B active parameters per token (blazing generation speed combined with 35B domain intelligence).
• Importance Matrix: 510 calibrated tensor entries derived from 550 dense chunks of statutory law, financial audits, and regulatory filings.
• Memory Footprint: Lean 13.63 GiB weight size leaving ample VRAM for deep document context windows.

---

Comparative Quantization Analysis (vs. Flat Quants & Generic APEX)

Also, don't confuse APEX-I-MiniPlus-V2 with a generic baseline APEX-I-Mini. Traditional APEX-I-Mini drops core experts aggressively to 2-bit IQ2_S and leaves output.weight at 3-bit Q3_K_M, which creates a noticeable perplexity hit on complex reasoning tasks. V2 was specifically re-engineered to avoid that quality floor (keeping core experts at calibrated IQ3_XXS, output in Q6_K, shared expert in non-linear IQ4_NL, and routers in F32).

To put the numbers in perspective: this cuts nearly 2 GB off a flat 3-bit quant (approx. 15.6 GB), and weighs only about approx. 1 GB more than a generic APEX-I-Mini (approx. 12.5 GB). For that single extra gigabyte of VRAM, you get a massive jump in reasoning and syntactic stability.

Take a look at the tensor-by-tensor comparison table below to inspect the exact architectural differences and see why this specific allocation is optimal. That's specifically what this was built for:

---

Bundled Q8_0 High-Precision Multimodal Vision Projector

Standard community uploads often omit the multimodal projector or supply uncompressed FP16 files (approx. 857 MB), bloating memory.

• Bundled Q8_0 Projector: Pre-quantized to Q8_0 (582 MiB / 610 MB), saving approx. 300 MB of VRAM.
• Audited Layer Fallbacks: llama.cpp automatically preserved 27 critical normalization and embedding tensors in F32/F16, ensuring razor-sharp OCR of tiny contract footnotes, financial balance sheets, and scanned legal filings without artifacts.

---

Everyday Laptop Benchmarks (DDR4 / DDR5 RAM)

Estimated Projections on Consumer Hardware

You do not need enterprise infrastructure to perform automated legal analysis. Estimated throughput projections on an everyday consumer laptop (Intel Core i5 / AMD Ryzen, 4GB/6GB Laptop GPU, 32GB DDR4/DDR5 RAM):

• GPU VRAM Allocation: Uses only approx. 3.8 GB VRAM (fits easily on budget laptop GPUs like RTX 3050, 4050, or 2060).
• System Memory Offload: Standard 32GB system RAM accommodates the remaining layers.
• Estimated Document Ingestion (Prefill): 300 to 420+ tokens/second sustained across dense briefs.
• Estimated Streaming Generation: 20 to 24+ tokens/second sustained output across system RAM!

---

The 24GB Miracle: Full 256K Context Runs In VRAM!

Legal and auditing tasks require ingesting hundreds of pages of case law, depositions, and regulatory exhibits. Automated 3-bit/4-bit community quants exceed 16–19 GiB in weights alone, crashing with Out-Of-Memory (OOM) errors when context scales up.

Thomson APEX-I-MiniPlus-V2 fits the entire 256K context window within 24GB VRAM:

Note: Projections leave approx. 4.65 GiB of headroom on 24GB cards for display buffers and the Q8 vision projector.

---

Hardware Throughput Projections (RTX 30 / 40 / 50)

| Hardware Target | Offload Mode | Generation Speed (Est.) | Prompt Prefill Speed (Est.) | Engineering Highlights | | :--- | :--- | :---: | :---: | : | | NVIDIA RTX 5080 / 5090 (Blackwell) | Full GPU (-ngl 99) | approx. 247 – 251 tok/s | 2,800 – 3,900+ tok/s | Empirically verified on RTX 5090 by zephel01 (Occamy V2 Reference) | | NVIDIA RTX 4090 (24GB GDDR6X) | Full GPU (-ngl 99) + mmproj | 75 – 100+ tok/s | 1,700 – 2,500+ tok/s | Real-time multi-page document review & synthesis | | NVIDIA RTX 3090 (24GB GDDR6) | Full GPU (-ngl 99) + mmproj | 62 – 78+ tok/s | 1,350 – 1,950+ tok/s | Full 256k legal brief ingestion in dedicated VRAM | | NVIDIA RTX 4080 / 5070 (16GB) | Partial offload (approx. 30 layers) | 32 – 42+ tok/s | 750 – 1,150+ tok/s | High-efficiency local legal workstation | | Consumer Laptop (4GB GPU + 32GB RAM)| Hybrid Offload | 20 – 24+ tok/s | 300 – 420+ tok/s | Smooth streaming from system DDR4/DDR5 RAM |

⚙️ Recommended Generation Parameters (Thomson Reuters Official)

Official generation parameters derived from the Thomson Reuters Technical Report for high-stakes professional analysis:

[!IMPORTANT]

🔍 Model Inherent Behavior vs. Quantization Fidelity Notice

Any behavioral nuances, stylistic tendencies, domain-specific habits, or zero-shot edge-case oversights stem entirely from the original unquantized checkpoint weights and fine-tuning distribution, NOT from the APEX-I quantization process. Handcrafted APEX-I-MiniPlus strictly preserves mathematical tensor fidelity—keeping 100% of expert routing matrices (gate_inp) in uncompressed F32 (zero router drift), armoring the token output head in Q6_K, and safeguarding attention gates in Q8_0. Empirical verification confirms near-zero perplexity loss (ΔPPL ≈ +0.05), ensuring that token logits, routing decisions, and reasoning trajectories are mathematically faithful to the original base model.

7. Community Compute Fund & Student Support

All IsValorum quantizations will always remain completely free and open to the public without paywalls.

However, cloud GPU compute is expensive and funded entirely out-of-pocket. If you find these handcrafted builds to be of high quality and they benefit your day-to-day workflow, you can support future Day-0 releases on Ko-fi. Every contribution directly funds the cloud compute hours required for new model runs and helps support my university education:

(When supporting on Ko-fi, feel free to leave a note with your Hugging Face handle and the specific model you would like prioritized).