APEX Quantization Series by IsValorum
[!IMPORTANT] Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.
If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.
Get the latest Windows installer · Release notes
Your model. Your PC. Real tools. V-Agent is a free, open-source Windows desktop agent developed by the author of these APEX quantizations. Run GGUF locally, or connect a supported remote API.
Open Models, search Inkling-2-Mini-APEX-I-NanoPlus-V2.1-GGUF, choose a compatible GGUF and confirm the download. V-Agent guides engine and model loading where supported.
| Tier | Footprint | Base | Abliterated |
|---|---|---|---|
| MiniPlus | 12.09 GiB | Base | Abliterated |
| NanoPlus | 10.23 GiB | Current | Abliterated |
thinkingmachines/Inkling-2-Miniinkling2_loopBF16ggml-org/Inkling-2-Mini-GGUF/imatrix.gguf (used file; external provenance not inferred)The table below is generated from the audited physical GGUF.
| Layer Group | Sub-Component / Tensor | Qty | Precision | Engineering Rationale |
|---|---|---|---|---|
| Global tensors | output.weight |
1 | Q6_K |
Audited global tensor |
| Global tensors | output_res_query.weight |
1 | F32 |
Audited global tensor |
| Global tensors | token_embd.weight |
1 | Q4_K |
Audited global tensor |
| Attention and gates | blk.{0..23}.attn_gate.weight |
24 | Q8_0 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_k.weight |
24 | Q4_K |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_k_conv.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{5..23:6}.attn_k_conv_proj.weight |
4 | Q8_0 |
Audited attention projection |
| Attention and gates | blk.{0..4,6..10,12..16,18..22}.attn_output.weight |
20 | Q4_K |
Audited attention projection |
| Attention and gates | blk.{5..23:6}.attn_output.weight |
4 | Q6_K |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_post_conv.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_q.weight |
24 | Q4_K |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_q_clip.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_rel.weight |
24 | Q4_K |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_rel_bias.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_rel_clip.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{1..23}.attn_res_query.weight |
23 | F32 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_sink_proj.weight |
24 | Q8_0 |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_v.weight |
24 | Q4_K |
Audited attention projection |
| Attention and gates | blk.{0..23}.attn_v_conv.weight |
24 | F32 |
Audited attention projection |
| Attention and gates | blk.{5..23:6}.attn_v_conv_proj.weight |
4 | Q8_0 |
Audited attention projection |
| Dense FFN / MLP | blk.{0..23}.ffn_post_conv.weight |
24 | F32 |
Audited dense MLP projection |
| Dense FFN / MLP | blk.{0..23}.ffn_res_query.weight |
24 | F32 |
Audited dense MLP projection |
| Routed experts | blk.{0..1,22..23}.ffn_down_exps.weight |
4 | Q5_0 |
Audited routed-expert projection |
| Routed experts | blk.{2..21}.ffn_down_exps.weight |
20 | IQ4_NL |
Audited routed-expert projection |
| Routed experts | blk.{0..1,22..23}.ffn_gate_up_exps.weight |
4 | IQ3_XXS |
Audited routed-expert projection |
| Routed experts | blk.{2..11}.ffn_gate_up_exps.weight |
10 | IQ2_S |
Audited routed-expert projection |
| Routed experts | blk.{12..21}.ffn_gate_up_exps.weight |
10 | IQ2_XXS |
Audited routed-expert projection |
| Routing and indexer | blk.{0..23}.ffn_gate_inp.weight |
24 | F32 |
Audited routing/indexer parameter |
| Normalizations | blk.{5..23:6}.attn_k_inp_norm.weight |
4 | F32 |
Verified physical normalization type |
| Normalizations | blk.{0..4,6..10,12..16,18..22}.attn_norm.weight |
20 | F32 |
Verified physical normalization type |
| Normalizations | blk.{5..23:6}.attn_q_inp_norm.weight |
4 | F32 |
Verified physical normalization type |
| Normalizations | blk.{5..23:6}.attn_v_inp_norm.weight |
4 | F32 |
Verified physical normalization type |
| Normalizations | blk.{0..23}.ffn_norm.weight |
24 | F32 |
Verified physical normalization type |
| Normalizations | output_norm.weight |
1 | F32 |
Verified physical normalization type |
| Normalizations | token_embd_norm.weight |
1 | F32 |
Verified physical normalization type |
| Other block components | blk.{0..23}.exp_probs_b.bias |
24 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_ffn_ctrl_down.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_ffn_ctrl_forget.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_ffn_ctrl_gain.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_k_ctrl_down.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_k_ctrl_forget.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_k_ctrl_gain.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_msg_query.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_q_ctrl_down.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_q_ctrl_forget.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_q_ctrl_gain.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_v_ctrl_down.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_v_ctrl_forget.weight |
12 | F32 |
Audited structural parameter |
| Other block components | blk.{6..17}.loop_v_ctrl_gain.weight |
12 | F32 |
Audited structural parameter |
thinkingmachines/Inkling-2-Mini