ATX-Swift-1.5-Qwen3.8-27B-Uncensored-MTP-GGUF
Custom ATX GGUF quantizations of Swift-1.5-Qwen3.8-27B-Uncensored-MTP featuring an embedded Multi-Token Prediction (MTP) draft head in blk.64, specifically optimized for high-throughput inference in llamAmpere and standard llama.cpp.
Optimized Engine: llamAmpere
While compatible with upstream llama.cpp, this model family is engineered for llamAmpere, a specialized high-performance fork tuned for NVIDIA Ampere (SM86) and Ada Lovelace architectures (RTX 3090 / RTX 4090).
Key Features Enabled in llamAmpere:
- MMA Fused Q8 Kernels (
GGML_Q8_TURBO3_MMA_FUSED="1"): Eliminates memory bandwidth bottlenecks and achieves peak Tensor Core utilization. - Hardware-Accelerated MTP Decoding (
--spec-type draft-mtp): Reaches 45-60+ tokens/s on a single RTX 3090 (24GB) via self-speculative draft verification. - Turbo3 Quantized KV Cache (
-ctk q8_0 -ctv turbo3): Unlocks 131k+ context lengths inside 24GB VRAM with zero context truncation. - Zero Warp Divergence: Non-linear quants (
IQ4_NL,IQ3_S) are omitted in favor of linear K-quants (Q5_K_M,Q6_K,Q8_0) to guarantee deterministic CUDA execution.
Model Lineage & Architecture
- Uncensored (Abliterated): Refusal directions neutralised across 131 core tensors (attention projections, MLP down-projections, and token embeddings).
- Swift 1.5 Architecture: Generates 58.5% fewer internal thinking tokens compared to base Qwen 2.5/3.0 models for 2x faster response delivery.
- Embedded MTP Head: 15 draft tensors located in
blk.64preserved at high precision (Q8_0) for speculative decoding draft acceptance rates above 80%. - Custom imatrix Calibration (i1): Calibrated against technical reasoning, code generation, and multi-turn conversational corpora.
Available Model Files
| File | Size | Target Hardware | Description |
|---|---|---|---|
ATX-Swift-1.5-Qwen3.8-27B-Uncensored-MTP-i1-Q8_0.gguf |
~29.0 GiB | 48 GB VRAM / Dual GPU | Near-lossless master precision. |
ATX-Swift-1.5-Qwen3.8-27B-Uncensored-MTP-i1-Q6_K.gguf |
~22.4 GiB | 32 GB VRAM | High-fidelity quantization. |
ATX-Swift-1.5-Qwen3.8-27B-Uncensored-MTP-i1-Q5_K_M.gguf |
~20.2 GiB | 24 GB VRAM (RTX 3090/4090) | Recommended Sweet Spot for llamAmpere. |
ATX-Swift-1.5-Qwen3.8-27B-Uncensored-MTP-i1-Q4_K_M.gguf |
~17.3 GiB | 20-24 GB VRAM | Fast and lightweight. |
mmproj-BF16.gguf |
BF16 | 0.9 GB | Vision projector, needed only for image and video input. |
Recommended Execution (llamAmpere on Windows PowerShell)
Clone and build the repository from GitHub: JakeATX/llamAmpere, then run:
cd \llamAmpere
$env:GGML_Q8_TURBO3_MMA_FUSED="1"
.\build-sm86\bin\Release\llama-server.exe `
-m "ATX-Swift-1.5-Qwen3.8-27B-Uncensored-MTP-i1-Q5_K_M.gguf" `
-ngl 99 `
-c 131072 `
-b 2048 `
-ub 512 `
-t 20 `
-tb 8 `
-fa on `
-ctk turbo5 `
-ctv turbo4 `
--kv-unified `
--prio 3 `
--parallel 1 `
--jinja `
--fit off `
--cache-prompt `
--cache-ram 4096 `
--spec-type draft-mtp `
--spec-draft-n-max 3 `
--spec-draft-p-min 0.1 `
--spec-draft-type-k q8_0 `
--spec-draft-type-v q8_0 `
--spec-draft-vocab-map "llamAmpere\docs\mtp-vocab\atx_65536.txt" `
--reasoning-format none `
--temp 0.2 `
--top-p 0.90 `
--top-k 40 `
--min-p 0.05 `
--repeat-penalty 1.08 `
--repeat-last-n 256 `
--alias "ATX-Swift-1.5-Qwen3.8-27B-Uncensored-Q5_K_M" `
--host 127.0.0.1 `
--port 8080
Upstream References Base Architecture & Cyber Abliteration: Qwen3.8-27B-Uncensored-Cyber High-Performance Inference Engine: llamAmpere