Qwen3.8 Flash Next

IQ4_XS · direct OrcaRouter · Measured configuration

83.42 / 100 Mango weighted score · 37.36 min matched workload · 12/12 mechanical agent passes

OrcaRouter's own three-shard IQ4_XS distribution of Qwen3.8 Flash Next, preserved with its exact source revision and matching tokenizer companion.

Benchmark results · Files · Download · Runtime · Provenance · Full collection

Join Infinity on Discord

Get model support, request models and quants, discuss benchmarks, and follow new releases and model improvements.

Join the Infinity community

All model tests, per-question scores, experiment history and scoring methods

Qwen effort study — separate full-404 cohort: medium 82.87/100 versus xhigh 84.41/100; paired difference +1.54 points (95% interval [-1.71, +4.82]). 404 collected questions, 403 valid questions graded per effort; one invalid reference annotation excluded symmetrically. The historical 202-question cross-model comparison below is unchanged. Study, timing and blind subjective results.

Preserved and documented by isilicon / Mango. The original model, modification and quantization remain the upstream creators' work. This repository adds attributed preservation and exact-configuration evaluation, with the selected source bytes unchanged.

At a glance

GB is decimal and GiB is binary. Payload size is disk/storage size, not a claim that the model fits in the same amount of VRAM. Runtime, KV cache, working memory and companions add their own requirements.

Mango evaluation

The exact three-shard package scored 83.42, with 19/20 local coding tests, 30/40 function-call checks and 12/12 mechanical agent passes.

It scored 0.77 points below the previous mradermacher package and took 37.36 rather than 35.33 workload minutes. The paired 95% difference interval crosses zero; this run did not demonstrate an upgrade.

Domain scores equally average their component task means. The overall score applies the weights above. EQ is continuous; the 12-case mechanical agent pass rate is only one component of the agentic domain.

Task-level quality and uncertainty

These are small released subsets and local graders. HumanEval/BFCL scores are not the complete official benchmark pipelines. Agent checks cover completion, evidence, permissions and applicable pricing totals; other answer semantics remain unjudged.

Recorded performance

Workload time excludes loading, restarts, wipe downtime and idle gaps. Latency/TTFT summaries cover suite requests only. The reported token rate divides engine usage counts by request time; it is not native decode throughput and token accounting differs across engines. No concurrent-serving result is implied.

Matched cohort comparison

The old mixed GLM row is a historical, internally capped baseline whose exact weights are unavailable. Other rows also differ in quant, engine, context and effort. Point ranks do not settle the quality ordering.

Paired score differences

Intervals use 10,000 paired task-stratified bootstrap resamples, seed 41, with fixed task/domain weights. An interval containing zero does not establish a quality winner. See methodology for scope and definitions.

Download the current full numerical report and methodology. The original compact evaluation is retained as the earlier preservation snapshot; its model score is unchanged, while the current report includes the expanded cohort. The numerical report contains aggregate statistics and hashes; no prompts or generated answers are published.

Runtime and compatibility

Requires qwen4_exp support. Mango tested llama.cpp revision c3df4b7e2, fixed 32K context, Q8 KV caches, thinking enabled and reasoning budget -1. The separate tokenizer/config mirror preserves the source-side companion files.

32,768 tokens; requested high mapped to native medium; no artificial reasoning budget.

The local evaluation covers text/tool tasks. It does not establish the full advertised context ceiling, image/audio quality or every optional speculative-decoding path. Publisher capabilities and Mango-verified behavior are separate scopes.

Companion artifacts

The selected payloads of this model and its linked companions total 97.49 GB before runtime/cache files. That is a download/storage total, not a RAM/VRAM fit guarantee.

Files and integrity

Full paths, byte sizes and complete SHA-256/Git blob identities are in SOURCE_MANIFEST.json. Available original README/license/notice files are also preserved. MIRROR_STATUS.json records the earlier source-file verification; MODEL_CARD_STATUS.json records this documentation refresh separately.

Download and use

The commands pin the already-verified mirror weight revision. Documentation can advance on main without changing that preserved weight snapshot. Keep original subfolders and all shards.

python -m pip install -U huggingface_hub

hf download isilicon/mango-flashnext-orca-iq4xs-20261029 \
  Qwen3.8-Flash-Next-Uncensored-IQ4_XS-00001-of-00003.gguf \
  Qwen3.8-Flash-Next-Uncensored-IQ4_XS-00002-of-00003.gguf \
  Qwen3.8-Flash-Next-Uncensored-IQ4_XS-00003-of-00003.gguf \
  --revision 2428954957bc6bbd5ba3d252de594abc155b2d12 \
  --local-dir ./weights/flashnext-orca-iq4xs-20261029

hf download isilicon/mango-flashnext-orca-tokenizer-20261029 \
  config.json \
  tokenizer.json \
  tokenizer_config.json \
  --revision 88209668ab21f0a3323ce5974e014f17bb988c3a \
  --local-dir ./weights/flashnext-orca-tokenizer-20261029

Load example and setup scope

llama-server \
  --model ./weights/flashnext-orca-iq4xs-20261029/Qwen3.8-Flash-Next-Uncensored-IQ4_XS-00001-of-00003.gguf \
  --host 127.0.0.1 --port 8080 --ctx-size 32768 \
  --jinja --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0

This is a minimal text-serving example for a compatible build, not a replay of every benchmark fitting/offload flag. The recorded configuration and engine caveats define the tested scope.

Provenance and license

This direct Orca package and the mradermacher package share the same upstream modification lineage. Their different files, quantization paths and runtime details must remain visible when comparing results.

Original notices, base-model conditions and the publisher's stated limitations still apply. Downloadable preservation does not add new license rights. Source-described refusal modification and publisher-side benchmark claims have not been independently re-measured by this Mango task cohort.

Original publisher documentation

The unchanged upstream README and pinned source card retain the original usage guidance, technical discussion, attribution, caveats and citations.

Read the complete original publisher card

Source reference. The following is the original publisher's documentation. Its other quants, full-model files, context claims and benchmark tables can exceed the selection in this mirror. Relative asset/file URLs are qualified to the pinned original source; publisher measurements remain separate from Mango measurements.

Qwen3.8-Flash-Next-Uncensored-GGUF

GGUF quants (2-bit → 8-bit) of the abliterated (refusal-removed) Qwen3.8-Flash-Next — for llama.cpp

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Ollama · Discord · X

---

GGUF conversions of the abliterated (refusal-removed) build of Qwen's Qwen3.8-Flash-Next — a large Mixture-of-Experts (512 experts, 10 routed + 1 shared active) preview of the Qwen4 architecture (qwen4_exp): Gated DeltaNet linear attention + Qwen Sparse Attention (QSA) at the micro-block level, HyperConnections in place of layer norms, PLE n-gram hash embeddings, native vision-language, reasoning, and tool-calling. These files run in llama.cpp (CPU / CUDA / Metal / ROCm), quantized from 2-bit to 8-bit, with a separate mmproj file that restores vision. Browse all models in the OrcaRouter Model Catalog.

---

Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). It will comply with harmful, unethical, or illegal requests the original Qwen3.8-Flash-Next would refuse. Released strictly for legitimate research — interpretability, AI-safety / refusal-mechanism study, red-teaming, and robustness evaluation. You assume full responsibility for how you use it and everything it generates; add your own safety and moderation layers before any deployment. Use must comply with the Apache 2.0 License inherited from the base model and all applicable law. The authors accept no liability for misuse.

---

Requirements — build a recent llama.cpp

The qwen4_exp architecture (Gated DeltaNet + QSA + HyperConnections + PLE n-gram) was merged into mainline llama.cpp on 2026-08-27 (PR #27742, model: add Qwen3.8-Flash-Next (qwen4exp), plus follow-up #27880). Any llama.cpp built from master at/after that date loads these files. Older or bundled runtimes that predate it will not — e.g. LM Studio's current build stops at qwen35moe and fails with unknown architecture 'qwen4_exp'; those will work once they bump their bundled llama.cpp. Build a current one yourself:

git clone https://github.com/ggml-org/llama.cpp/
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first \
  --target llama-cli llama-mtmd-cli llama-server llama-gguf-split

(Drop -DGGML_CUDA=ON for a CPU-only build.)

MTP speculative decoding (new). As of PR #30592 (stacked on #27836), llama.cpp's qwen4exp backend has a real multi-token-prediction (mtp) inference path. All quants here — including Q8_0 — are plain target-only files that load on any recent mainline llama.cpp / LM Studio. To add speculative decoding, pair any quant with the small …-MTP-draft.gguf (~4 GB, in Files) via -md --spec-type draft-mtp on a #30592 build (see Usage). A separate …-Q8_0-MTP build (the MTP draft head embedded in the same file, so it self-speculates without -md) is also provided for #30592 users — it needs that build and will not load on mainline. Apple Silicon users can also use the MLX build orcarouter/Qwen3.8-Flash-Next-Uncensored-MLX, which ships the mtp/ drafter.

Files

Files larger than ~48 GB are split with llama-gguf-split into multiple parts — download the whole set for that quant and point llama.cpp at the …-00001-of-000NN.gguf part; it loads the rest automatically. Sizes are approximate.

Standard K-quants

Q6_K and higher. The model's n-gram (PLE) embedding table is a single 51.2B-parameter tensor (per_layer_token_embd, shape [160, 320001536]; 102.4 GB in BF16). Its quantized dimension (160) is not divisible by 256, so it cannot be a K-quant — at Q6_K it falls back to Q8_0 (~54.4 GB) and occupies its own shard. That shard is above CloudFront's 50 GB single-request download limit (a delivery constraint, not a storage limit), so it is served via Xet / ranged requests, which the default hf client and recent llama.cpp downloaders handle transparently — the same approach upstream Unsloth uses for its Q6_K/Q8_0/BF16 builds. Q6_K and Q8_0 are provided above (a separate Q8_0-MTP variant embeds the MTP draft head); BF16 is omitted as impractically large — for full precision use the BF16 safetensors in orcarouter/Qwen3.8-Flash-Next-Uncensored.

IQ quants (imatrix)

Lower-bit quants built with an importance matrix (computed on English + Chinese + code calibration text) — better quality-per-bit than plain K-quants at the low end, especially IQ3/IQ2.

Vision

Speculative draft (MTP)

Usage (llama.cpp — PR #27742 build)

Download

hf download orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF \
  --include "Qwen3.8-Flash-Next-Uncensored-Q4_K_M*" "mmproj-*" \
  --local-dir ./qwen-flashnext-uncensored

Chat (text)

./llama-cli -m Qwen3.8-Flash-Next-Uncensored-Q4_K_M-00001-of-00003.gguf --jinja -c 8192 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 -p "Hello!"

OpenAI-compatible server (tool calling + reasoning + vision)

./llama-server -m Qwen3.8-Flash-Next-Uncensored-Q4_K_M-00001-of-00003.gguf \
  --mmproj mmproj-Qwen3.8-Flash-Next-Uncensored-F16.gguf \
  --host 0.0.0.0 --port 8000 -c 8192 --jinja

• Recommended sampling: --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0.
• Vision: pass --mmproj …, then send OpenAI image_url content parts (base64 data-URI or URL).
• Tool calling: --jinja enables the Qwen tool template; use standard OpenAI tools + tool_calls.
• Reasoning (thinking): thinking is on by default; toggle per request via chat_template_kwargs.enable_thinking. The reasoning trace is returned in reasoning_content — give max_tokens room (e.g. >= 2048) so the final answer isn't truncated by the thinking budget.

Speculative decoding with MTP (Q8_0-MTP / IQ4_XS-MTP, PR #30592 build)

The Q8_0-MTP build embeds the model's multi-token-prediction draft head. On a llama.cpp built from PR #30592, turn on single-file self-speculative decoding — no separate draft model required:

./llama-cli -m Qwen3.8-Flash-Next-Uncensored-Q8_0-MTP-00001-of-00005.gguf --jinja -c 8192 --spec-type draft-mtp --spec-draft-n-max 3 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 -p "Hello!"

Keep --spec-draft-n-max small (2–3) for best acceptance. This path exists only on the #30592 build; the plain Q8_0 (and every other quant) loads on mainline llama.cpp as usual. For MTP on mainline-compatible files, use any quant with the standalone -md draft below.

The same single-file self-speculation also works with the much smaller IQ4_XS-MTP (~100 GB, the IQ4_XS weights with the MTP head embedded, ~0.77 draft acceptance):

./llama-cli -m Qwen3.8-Flash-Next-Uncensored-IQ4_XS-MTP-00001-of-00003.gguf --jinja -c 8192 --spec-type draft-mtp --spec-draft-n-max 3 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 -p "Hello!"

For any other quant (Q2_K–Q6_K, IQ*), pair it with the standalone MTP draft via -md:

./llama-cli -m Qwen3.8-Flash-Next-Uncensored-Q4_K_M-00001-of-00003.gguf -md Qwen3.8-Flash-Next-Uncensored-MTP-draft.gguf --jinja -c 8192 --spec-type draft-mtp --spec-draft-n-max 3 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 -p "Hello!"

Evaluation

Abliteration was measured on this build (served with vLLM) vs the official Qwen/Qwen3.8-Flash-Next, using the same scripts. Harmful-prompt refusal collapses from 64–100% (base) to ~0–3.3%; benign over-refusal stays near 0%; capability stays within ±2 pts of the base across MMLU-Pro / GSM8K / CMMLU-style checks; vision (image + OCR via mmproj) and multi-turn tool calling verified working. GGUF quants are deterministic derivations and inherit these behaviours; lower bits trade some quality (most visible at Q2_K / IQ2).

Hardware

Runs on CPU, CUDA, Metal, or ROCm via llama.cpp. As a large MoE, only ~10 of 512 experts are active per token, so decode is far faster than the total parameter count suggests, but the full weights must fit in RAM/VRAM (or be memory-mapped): budget ≈ the file size + KV cache + (for vision) the ~0.9 GB mmproj. Multi-GPU (-ngl split) and CPU+GPU offload both work.

License

Apache 2.0, inherited from Qwen/Qwen3.8-Flash-Next. Abliteration and quantization do not change the underlying license obligations.

Preservation record

Selected source bytes are verified against pinned metadata; the original eight graded mirrors also passed the previously recorded anonymous 1 MiB weight-prefix checks. This is not a full independent-device restore/load test. Hugging Face repository copies share a provider and do not guarantee permanent storage.

Collection · Source manifest · Original documentation