An uncensored (abliterated), all-NVFP4 Qwen3.8-27B artifact for the
NInfer engine, built end-to-end on a single
RTX 5090: d0xin's abliterated BF16 checkpoint → our own NVFP4 quantization
pipeline → DFlash2 speculative drafter + indexed proposal head, packaged as a
native v3 .ninfer container.
It is the uncensored counterpart to kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3 and replaced it as our production profile after a same-session A/B (below).
This artifact exists only because of the following models and their authors — all credit to them:
| Role | Model | Author | Commit / SHA-256 |
|---|---|---|---|
| Abliterated BF16 source (the weights we quantized) | d0xin/Swift-Qwen3.8-27B-Uncensored-BF16 | d0xin | config.json 191e0af2…, ABLITERATION.json c0ffdb6d…, model-00001-of-00018 2099c92d…, model-00018-of-00018 dc8df5ce… (full list in the repo's RELEASE_HASHES.txt, all verified before quantization) |
| Upstream base checkpoint | ukisai/Swift-Qwen3.8-27b | ukisai | — |
| Abliteration technique lineage | huihui-ai/Huihui-Qwen3.8-27B-abliterated | huihui-ai | refusal-direction removal family (Arditi et al. 2024) |
| DFlash2 speculative drafter | z-lab/Qwen3.8-27B-DFlash2 | z-lab | 50307d4c4cde6860d4eee73e2547cd786fe8e8a4 |
| Uncensored-evaluation reference | Pliny the Liberator's OBLITERATUS suite & Barding-Defense's ninferization walkthrough | quantization-runbook inspiration | |
| Engine | Neroued/ninfer | Neroued | v3 container, ≥ 98dada0e |
The abliteration itself is d0xin's work, inherited byte-for-byte — no
re-abliteration, no behavioural editing was performed here. Per
ABLITERATION.json: OrcaRouter-compatible rank-1 residual-writer directional
ablation, W' = W − r(rᵀ W), 131 modified tensors, direction layer reference
38, rank-1, fp32 projection math, max BF16 residual leakage 1.49e-3. d0xin's own
paired evaluation (298 samples) measured MMLU-Pro +6.1 pts vs base (not
significant, CI crosses 0) and math-500 unchanged.
llm-compressor
(512 Ultrachat calibration samples, seq 2048, sequential pipeline):
NVFP4 W4A4 group-16 on every text projection — MLP gate/up/down (all 64
layers), full-attention Q/K/V/O, GDN in_proj_qkv/z/out_proj — with the
9-tensor BF16 exception pattern of the registered Qwen3.6-27B profile;
token embedding + output head in W8G32 groupwise (matching the cometkim
nvfp4full profile); vision tower on the official q6/q8 allocation; MTP and
DFlash2 drafter carried in BF16.weight_global_scale; the NInfer engine's native A4 input
route requires each layer's fused attention parent (GDN qkvz = 16,384 rows,
full-attn qkgv = 14,336 rows) to be one contiguous region with a single
divisor. We unify each packing group's divisor to D = min(dᵢ) and rescale
every member's E4M3 block scales by D/dᵢ (RNE, shrink-only → overflow-free).
Verified dequant-invariance: worst drift 2.7% on the smallest-magnitude
elements, 0.0000% median — weights unchanged up to E4M3 re-encode noise.
With uniform divisors the converter coalesces each group into one fused
parent and the engine accepts the artifact unmodified.tools/convert @ fork 3090a5b1, 2026-09-24 rebase of upstream),
--components text,vision,mtp,dflash2 --proposal.| Field | Value |
|---|---|
| Filename | qwen3_8_27b_swift_abliterated_nvfp4full-dflash2.ninfer |
| Size | 19,782,447,364 bytes (18.42 GiB) |
| SHA-256 | 74c9721303f5287bf3fbd39544f0a6446b03a95b7c28b3751d5e12866ca8b918 |
| Container version | 3 (NINFER\0\x03) |
| NInfer model ID | qwen3.8-27b |
| Stored objects | 1,590 (Text + Vision + MTP + DFlash2 + indexed proposal head) |
| Formats | nvfp4 ×348 (fused text parents), q8_g32 ×30, q4/q5/q6 (vision), bf16 remainder |
| Full conversion contract | qwen3_8_27b_swift_abliterated_nvfp4full-dflash2.ninfer.conversion.json (in this repo) |
| Hash manifest | SHA256SUMS |
ninfer-serve qwen3_8_27b_swift_abliterated_nvfp4full-dflash2.ninfer \
--host 0.0.0.0 --port 8080 --model-id Qwen3.8-27B \
--max-context 262144 --kv-capacity auto --kv-dtype k8v4 \
--max-concurrency 4 --default-max-tokens 32768 \
--pending-timeout-ms 900000 --prefill-chunk 4096 \
--temperature 0.9 --min-p 0.05 \
--spec dflash2 --draft-tokens 7 --lm-head-draft \
--host-kv-mib 49152 --host-state-slots 16 \
--max-private-continuations 16 --max-shared-prefixes 16 \
--vision --default-thinking-budget 16384 --preserve-thinking
Stock-engine note:
--image-token-budgetand thedflash2speculative backend exist only in our forked engine build. On the stock NInfer engine, drop both (--image-token-budget 1280and--spec dflash2 --draft-tokens 7 --lm-head-draft): vision input is capped at 32,768 tokens by a compile-time constant, and the stockdflashbackend cannot be combined with--vision. The model runs fine without speculative decoding. The image-token-budget flag is upstream PR Neroued/ninfer#61 (open) — it lands in the stock engine when that PR merges; the fork build carries it in the meantime.
Live capacity on one 5090: weights 18.0 GiB, device KV pool 308,736 tokens (auto), k8v4, 48 GiB pinned host-KV arena, 262,144-token max context at concurrency 4. Full context is verified working (below).
Both sides served by the same engine build, same port, fresh generations, temp 0, no cache reuse, RTX 5090 @ 450 W / SM clock pinned 2280 MHz.
| Benchmark | Prior prod (nvfp4full-dflash2.v3) |
This artifact | Δ |
|---|---|---|---|
| IFBench prompt-strict (n=300) | 65.0 | 66.3 | +1.3 |
| IFBench prompt-loose | 67.7 | 70.3 | +2.7 |
| IFBench instr-strict | 64.5 | 67.7 | +3.2 |
| IFBench instr-loose | 66.9 | 71.2 | +4.4 |
| GSM8K-200 accuracy | 96.5% (193/200) | 95.5% (191/200) | −1.0 pp (within n=200 noise) |
| Gate decode (256-tok samples) | 146.4 tok/s | 148.4–159.0 tok/s | +1.4% or more |
| Perf decode (mean of 3) | 146.4 tok/s | 148.7 tok/s | +1.6% |
| Prefill @ 200k ctx | 2,911 tok/s | 3,260 tok/s | +12% |
| Weights in VRAM | 18.7 GiB | 18.0 GiB | −0.7 GiB |
| Artifact size | 20.55 GiB | 18.42 GiB | −2.1 GiB |
| Device KV pool (auto) | 278,912 tok | 308,736 tok | +29,824 tok |
| Long-context recall | not measured | 250,031 tokens: EXACT @ 10/50/90% depth | new evidence |
| Uncensored | no | yes | the point |
Long-context recall method: needle-in-a-haystack, haystack 1,000,000 chars (~250k tokens), secret at three depths, exact-match verdicts, zero corrupted / one transient empty response at 75k (passed on re-run).
The prior profile stored attention in heavier formats and carried a 2.23 GB
W8G32-encoded drafter. This build quantizes the entire text backbone (attention
included) to NVFP4 behind fused parents — NVFP4 tensor-core GEMMs being the
fastest path on Blackwell — and the freed VRAM is handed to the device KV pool
by --kv-capacity auto (+29,824 tokens ≈ 0.7 GiB). Abliteration itself adds
zero size: it is a rank-1 weight edit, no new parameters.
temperature 1.0 the model occasionally emits duplicate tool calls or a
short repetition loop in streaming tool-call tests (all emitted calls remain
valid JSON; the streaming parser handles them). At temperature 0 this has
never been observed across many runs; production sampling (temp 0.9,
min-p 0.05) suppresses it.All numbers on this card were measured on a single RTX 5090 32 GB (driver 595.71.05):
nv-power-limit.service systemd unit — a thermal-efficiency mod (load power
~450 W → ~330 W) with no measurable tok/s loss vs stock.ukisai/Swift-1.5-Qwen3.8-27b.pp8192 @ dN = throughput of 8,192 new tokens processed on top of N cached
context tokens — the agentic-coding profile (long, growing context; small
incremental prompts). e2e ttft is end-to-end time to first response chunk.| test | t/s (mean ± std of 3) |
|---|---|
| pp2048 | 48,325 ± 445 |
| tg256 | 240 ± 13 |
| pp2048 @ 16k cached ctx | 387,112 ± 3,950 |
| tg256 @ 16k cached ctx | 191 ± 17 |
| pp8192 @ 32k cached ctx | 746,186 ± 1,829 |
| tg512 @ 32k cached ctx | 217 ± 17 |
| pp8192 @ 131k cached ctx | 1,414,952 ± 2,835 |
| tg512 @ 131k cached ctx | 174 ± 20 |
| pp8192 @ 200k cached ctx | 1,737,206 ± 40,619 |
| tg512 @ 200k cached ctx | 163 ± 8 |
All artifacts are NInfer containers of Qwen3.8-27B derivatives, served by the same engine on the same box. Per-card A/B tables were measured pairwise in the same session; cross-card rows are from different sessions — treat small deltas as indicative.
| artifact | base | uncensored | IFBench prompt-strict | GSM8K-200 | decode tok/s | tg512 @ 131k ctx | status |
|---|---|---|---|---|---|---|---|
| nvfp4full-dflash2 (v2) | Qwen3.8-27B | no | — | — | 146.4* | — | superseded by v3 (same weights, v2 container) |
| nvfp4full-dflash2.v3 | Qwen3.8-27B | no | 65.0* | 96.5%* | 146.4* | 162.9 | available |
| swift-abliterated v3 | d0xin Swift-1.0 (huihui-abliterated) | yes | 66.3 | 95.5% | 148.7 | 173.7 | available |
| swift15 v3 | ukisai Swift-1.5 | no | 69.0 | 95.0% | 160.8 | 153.4 | rollback profile |
| thinkingcap v3 | BottleCapAI ThinkingCap | no | 68.0 | 95.5% | 167.6 | 177.3 | available (PolyForm — non-commercial) |
| swift15-uncensored-ajgazin v3 | ukisai Swift-1.5 + ajgazin/orcarouter ablation | yes | 70.3 | 94.5% | 165.7 | 187.9 | current production |
* values for the nvfp4full-dflash2.v3 row come from its same-session A/B against swift-abliterated (66.3-vs-65.0 etc. are paired measurements, not independent runs).
By downloading, possessing, or using this artifact you agree to the terms below.
Research, personal/local experimentation, red-teaming, safety research, and evaluating alignment-removal techniques — subject to all applicable laws and regulations.
You must not use this model, alone or in any pipeline, to:
This model is a capability, not a judgment. Abliteration removes refusal
behavior; it does not make outputs trustworthy, factual, or legal. You are
solely and fully responsible for every prompt you send, every output you
generate, and every use you make of them. The publisher of this artifact
(kaushikvira) does not monitor, endorse, or take any part in downstream use.
The artifact is provided "AS IS", WITHOUT WARRANTY OF ANY KIND, express or implied, including merchantability, fitness for a particular purpose, and non-infringement. To the maximum extent permitted by applicable law, the publisher shall not be liable for any claim, damages, or other liability, whether in contract, tort or otherwise, arising from, out of, or in connection with this artifact or its use. Nothing in this card limits liability where limitation is not permitted by law.
If you become aware of misuse of this model, report it to the platform where the misuse occurs and to the relevant authorities.
NInfer does not support JSON-schema output; if you need structured output or the standard vLLM/SGLang stack, use the HF-format NVFP4 weights:
If you benchmark an HF-format variant on your hardware, please share numbers in the repo's Community tab — we collect them on the variant's card.
Apache-2.0, inherited from the base artifact chain. Abliteration and base-model licenses: see the credited repositories above — all credit to d0xin, ukisai, huihui-ai, z-lab and Neroued.
d0xin/Swift-Qwen3.8-27B-Uncensored-BF16