ABLITERATED V6 T21 APEX MIT

Ling-3.0-flash-abliterated-APEX-GGUF

inclusionAI Ling-3.0-flash · 124B-A5.1B hybrid MoE · abliterix v6 trial 21 · APEX GGUF

⚡ About this release

Ling-3.0-flash is inclusionAI's next-generation native hybrid reasoning model: 124B total / 5.1B active (~12.4% of their previous 1T-class flagship). It uses a native hybrid-linear stack from pretraining — 5:1 Kimi Delta Attention (KDA) + gated MLA, 1/64 sparse MoE, 512 routed experts (top-8) + 1 shared expert, 2 dense layers, and a trained MTP head. Official context schedule is 8K → 32K → 256K. Thinking is on by default in the official card.

This community package is not an official inclusionAI release. It applies two extra steps on top of the official weights:

Refusal edit (mergeable recipe) via abliterix v6 trial 21 LoRA, stream-merged back to BF16. Write path: MPOA weight_normalization=full, LoRA rank 3, o_proj + down_proj only. This is the static merge, not the later runtime-only gate (v36).
GGUF: BF16 convert + APEX mixed-precision tensor-type files (no imatrix on these tiers). Architecture in the GGUF is bailingmoe3.

License: MIT (same family as the official model). Follow inclusionAI terms and local law.

⚠️ Abliterated notice

After merging abliterix v6 trial 21, this model shows a much lower refusal rate and can differ from official Ling-3.0-flash. Evaluate compliance and safety for your use case; control access and audit as needed.

Refusals (search eval)	17 / 98 (17.3%)
KL divergence	0.0933 nats/token
Selected trial	abliterix v6 trial 21 (mergeable). Not v36 (runtime gate).
Compact smoke (thinking off)	Capability ~9/10; benign refuse 0/10; harmful refuse ~2–3/40. Ablation holds. Thinking-on eval can eat the token budget — use `--reasoning off` or a larger `max_tokens`.

Implementation sketch: LoRA merge W += (B @ A) * (alpha / r) with MPOA full-norm rank 3 on o_proj + down_proj. Adapter path: Ling-3.0-flash-LoRA-Trial21-Refusals17-KL0.093.

🧠 Model details

Architecture	Hybrid-linear MoE (`bailing_hybrid` / GGUF `bailingmoe3`)
Parameters	124B total, 5.1B active / token
Layers	35 KDA + 7 gated MLA (5:1) · 2 dense · GGUF 42 backbone + 1 MTP (`blk.42`)
Experts	512 routed + 1 shared, top-8
Attention	KDA (fine-grained diagonal gating) + gated MLA, 32 heads, hidden 2560
Context	Trained 8K → 32K → 256K (practical limit depends on VRAM / `-c`)
Vocab	157,184
Modality	Text → text (no mmproj)
This repo	BF16 + APEX Compact / Quality / Balanced GGUF

Coding / agent ability is largely retained on Compact smoke; refusal and alignment behavior are changed. Official SWE / Tau / MCP tables were not re-run for this derivative.

💡 What is APEX?

These files use APEX-style MoE-aware mixed precision: precision follows tensor role + layer position (higher on edges, more aggressive in the middle).

This package is not I-tier (no imatrix). Common settings: 43 GGUF blocks (42 backbone + MTP blk.42); first 2 layers dense; routers ffn_gate_inp=Q8_0; MTP + nextn.eh_proj forced Q8_0; KDA/MLA tensor names as in llama.cpp bailingmoe3.

📦 Files

File	Size	Mid experts	Best for
`*-APEX-Quality.gguf`	75.90 GiB	edge Q6_K / near Q5_K / mid iq4_xs; shared Q8_0; attn Q6_K	Highest APEX quality (IQ mid-layers)
`*-APEX-Balanced.gguf`	85.96 GiB	edge Q6_K / near & mid Q5_K; shared Q8_0; attn Q6_K	Recommended default — steadier than Compact (no IQ mid)
`*-APEX-Compact.gguf`	56.25 GiB	edge Q4_K / mid Q3_K; shared Q6_K; attn Q4_K	Fits ~64–96 GB unified / VRAM with context room
`*-BF16.gguf`	237.57 GiB	native BF16 (norms / router / KDA state F32)	Requantize / archive. Too large for a 128 GB box.

How to choose: start with Balanced if you have the RAM. Use Compact on 128 GB UMA (this is what we smoke-tested). Quality if you want IQ mid-experts and a smaller file than Balanced. Skip BF16 unless you are converting again.

🚀 Usage (llama.cpp)

These GGUFs are general.architecture = bailingmoe3. BailingMoE3 support (PR #26608) was merged into llama.cpp master on 2026-08-17 — the first release containing it is b10470. Use llama.cpp b10470 or newer (any release or master build from that date onward) to load these files directly. If you see unknown model architecture: 'bailingmoe3', your build is older than that — update. Do not mix with unofficial bailing-hybrid forks — the tensor names do not match.

Example (Compact, thinking off)

hf download SC117/Ling-3.0-flash-abliterated-APEX-GGUF \ --include "Ling-3.0-flash-abliterated-APEX-Compact.gguf" \ --local-dir ./models

./llama-server
-m ./models/Ling-3.0-flash-abliterated-APEX-Compact.gguf
--port 8080
-sm none
--flash-attn on
--fit on
--ctx-size 32768
--reasoning off
--jinja
--host 0.0.0.0

Official sampling: temperature=0.6, top_p=0.95, top_k=20. Official HF/SGLang/vLLM path enables thinking by default; for llama.cpp eval we used --reasoning off so thinking does not eat the token budget.
MTP is present as blk.42. This package was served with --spec-type none; speculative decode needs a build that understands Ling MTP.
Text-only GGUF; no mmproj.

🎛️ Recommended sampling

Official default	temperature 0.6, top_p 0.95, top_k 20
Thinking	Official: on by default. llama.cpp: `--reasoning on\|off` as needed.

🔧 Build pipeline (summary)

Base: inclusionAI/Ling-3.0-flash
abliterix v6 search → trial 21 LoRA (o_proj + down_proj, MPOA rank 3)
BF16 merge → Ling-3.0-flash-abliterated
llama.cpp PR #26608 convert → BF16 GGUF
APEX tensor-type-file (no imatrix) → Compact / Quality / Balanced

Disclaimer

Community derivative (behavior edit + quantization). Not an official inclusionAI release. Use at your own risk; follow local law and the upstream MIT license.

SC117/Ling-3.0-flash-abliterated-APEX-GGUF

Ling-3.0-flash-abliterated-APEX-GGUF

Links

Disclaimer