Quick Navigation Index
Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.
If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.
- Model Overview & Technical Specifications
- Importance Matrix Optimization
- Surgical Quantization Topology (779 Tensors)
- Recommended Server & Inference Configuration
- Community Compute Fund & Student Support
1. Model Overview & Technical Specifications
This repository provides Flint APEX-I-NanoPlus V2.1, an ultra-efficient surgical quantization of RecursiveSI/flint.
Flint is Recursive's mixture-of-experts language model for coding agents, featuring 39.8 billion total parameters and approximately 2.5 billion active per token. It is specifically designed to investigate codebases, fix bugs, implement complex features, and write comprehensive unit tests.
The architecture features 49 layers: two initial dense SwiGLU MLP layers (blk.0 and blk.1) followed by 47 MoE layers (blk.2 to blk.48) with 256 routed micro-experts (top-k=6) and 1 shared expert. In addition, Flint incorporates learned attention sinks across all 49 layers for streaming stability over long contexts up to 262,144 tokens. The model topology encompasses exactly 779 tensors in GGUF.
NanoPlus V2.1 is engineered to hit the 13 GB footprint target (12.33 GiB in VRAM / 13.24 GB on Hub) for maximum memory bandwidth efficiency and high throughput on 16 GB GPUs with large context windows or partial CPU offloading on 12-16 GB setups. It strictly shields routing matrices, attention sinks, and normalization layers in full 32-bit floating point (F32), while strategically compressing interior routed experts with empirical Importance Matrix guidance.
Key Characteristics
- Base Model:
RecursiveSI/flint - Architecture:
FlintForCausalLM(Dense-Front + MoE + StreamingLLM Sinks) - Total Parameters: 39.79 Billion
- Active Parameters: approx. 2.5 Billion per token
- Layers: 49 (Layers 0-1 Dense SwiGLU, Layers 2-48 MoE)
- Routed Experts: 256 per layer (Top-k = 6)
- Shared Experts: 1 per MoE layer (intermediate size 1024)
- Context Window: 262,144 tokens (Sliding Window 513 with Attention Sinks)
- GGUF Tensor Count: Exactly 779 tensors verified
- Quantization Profile: APEX-I NanoPlus V2.1 (approx. 12.33 GiB in VRAM / approx. 13.24 GB on Hub)
2. Importance Matrix Optimization
Quantizing a 256-expert MoE architecture without an Importance Matrix results in routing instability, expert starvation, and severe degradation in long-horizon reasoning.
This release utilizes a comprehensive Importance Matrix calibrated by Bartowski (RecursiveSI_flint-imatrix.gguf), providing full 4-moment statistical distributions across all 531 layer components:
- First Moment / Mean Activation (
.in_sum): Allowsllama-quantizevector codebook optimizers (IQ2_S,IQ2_XXS,IQ3_XXS,Q4_K) to compute centered variance ($\sigma^2 = \frac{1}{N}\sum x^2 - \mu^2$) rather than uncentered raw energy, eliminating distortion on biased activations. - Extended Distribution Statistics (
.stats[12]): Prevents outlier clipping across routed expert weights. - Chat-Template Aligned Calibration: Tokenized with Flint's native
<|system|>...<|user|>...<|assistant|>chat template.
3. Surgical Quantization Topology (779 Tensors)
Every single tensor in Flint has been mapped according to the updated APEX-I V2.1 specification:
| Structural Component | Tensor Count | Precision Tier | Technical Rationale |
|---|---|---|---|
output.weight |
1 | Q6_K |
Preserves vocabulary logit fidelity and generation nuances |
output_norm.weight |
1 | F32 |
Final sequence normalization numerical shield |
token_embd.weight |
1 | Q4_K |
Robust input token embedding representation |
blk.*.attn_sinks.weight |
49 | F32 |
CRITICAL: Sliding-window initial sink token stabilization |
Normalizations (attn_norm, attn_q_norm, attn_k_norm, ffn_norm) |
196 | F32 |
Numerical stability and layer dynamic range protection |
blk.*.ffn_gate_inp.weight (MoE Routers) |
47 | F32 |
Strictly shields gating probabilities to prevent router collapse |
Dense FFN (blk.0 & blk.1: ffn_gate, ffn_up, ffn_down) |
6 | Q4_K |
Solid trunk representations before MoE branching |
Shared Experts (blk.*.ffn_*_shexp.weight) |
141 | Q4_K |
Preserves foundational representations across all tokens |
Attention Projections (attn_q, attn_k, attn_v) |
147 | Q4_K |
Fast and accurate key-query attention routing |
Attention Output Projections (attn_output) |
49 | Q6_K |
High-precision attention output projection across all 49 layers |
Boundary Routed Experts (4 early: blk.2..blk.5, 4 late: blk.45..blk.48) |
24 | IQ3_XXS |
Balanced boundary expert protection |
Core Routed Experts (blk.6..blk.44, 39 layers) |
117 | IQ2_S (down/gate) / IQ2_XXS (up) |
Optimal compression density hitting the 13 GB target |
| Total Verified Tensors | 779 | -- | 100% Accounting Verified |
4. Recommended Server & Inference Configuration
Runtime Requirement
Requires llama.cpp build b11803 or higher (e.g. b11821+) which incorporates Pull Request #30928 adding native support for FlintForCausalLM and sliding-window attention sinks.
CLI Launch Command
llama-server \
-m Flint.APEX-I-NanoPlus-V2.1.gguf \
--host 0.0.0.0 \
--port 8080 \
-c 32768 \
-ngl 99 \
--temp 1.0 \
--top-p 0.95 \
--top-k 20
5. Community Compute Fund & Student Support
All APEX-I and VAL-APEX quantizations are produced independently using rented cloud GPU/CPU clusters and validated through tensor-level accounting.
If you find this model useful in your research or production environment, please consider supporting future releases: