Qwen3.8-27B AEON Mixed FP8

Checkpoint size: 32.92 GB (30.66 GiB) of safetensors weights.

High-quality mixed E4M3 block-128 FP8 quantization of AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16, designed and validated for 4x NVIDIA Tesla V100 32GB (SM70) with 1Cat-vLLM.

This is a community quantization. It is not an official Qwen FP8 checkpoint.

Precision policy

• 336 large GEMM weights use dynamic-activation E4M3 FP8 with 128x128 weight blocks.
• 863 important tensors remain bit-identical to the BF16 source, totaling 10,264,372,704 bytes.
• The complete vision tower (333 tensors) remains BF16.
• The native MTP branch (15 tensors) remains BF16.
• Q/K/V/O in all 16 full-attention layers (64 tensors) remain BF16.
• Embeddings, LM head, norms, and state-sensitive GDN/SSM parameters remain BF16.
• Recommended V100 runtime precision is FP16 compute with FP16 KV cache.

The source revision is pinned to AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16@8f76e82ed7ef4de7735f5d4148fce7b643b00fae. The included PRECISION-MANIFEST.json, SOURCE-MANIFEST.json, FP8-ERROR-METRICS.jsonl, and artifacts/STATIC-AUDIT.json describe the build.

V100 validation

Validation used 4x Tesla V100-PCIE-32GB, TP4, one sequence, a 262,144-token window, FP16 compute, FP16 KV, and 1Cat-vLLM v0.1.dev17347+gc8a1a528e (main commit c8a1a528). Long requests were confirmed to hit the SM70 D=256 optimized attention path instead of a silent fallback.

Bounded paired evaluation against the BF16 source produced:

There was no reproduced quantization-only quality regression in this bounded suite. This is not a claim of equivalence over every task, language, sampler, or context distribution.

MTP0 versus MTP4

MTP0 is the recommended general-purpose V100 profile. MTP4 was validated and can substantially accelerate decoding when draft acceptance is high, especially for short prompts and longer outputs, but it also reduced prefill throughput by 3.7%-21.2% in the measured matrix. It was slower end-to-end for several long context cases. Do not enable MTP4 universally without workload-specific testing.

These FP16-KV results are not directly comparable to 1Cat's published E5M2 FP8-KV curve. FP8 KV is intentionally outside this release's validated quality profile because long-context quality was prioritized over additional KV savings.

Serving with 1Cat-vLLM

The validated MTP0 launch used the equivalent core arguments below. Adjust the batch-token budget and memory utilization for your host, and use a 1Cat-vLLM build that includes its SM70/Flash-V100 extensions.

vllm serve leoncca/Qwen3.8-27B-AEON-Mixed-FP8 \
  --trust-remote-code \
  --dtype float16 \
  --attention-backend FLASH_ATTN_V100 \
  --tensor-parallel-size 4 \
  --gpu-memory-utilization 0.85 \
  --kv-cache-dtype auto \
  --max-model-len 262144 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 15680 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3

On this validated stack, --kv-cache-dtype auto selected FP16 KV because the model runtime dtype was FP16. Confirm the actual dtype in your own startup logs.

Integrity

The three weight shard hashes are:

SHA256SUMS covers the complete published repository view.

Limitations and responsibility

This release inherits the capabilities and risks of the uncensored AEON source. The model may produce inaccurate, unsafe, illegal, or otherwise harmful content. Users are responsible for prompts, outputs, downstream actions, applicable legal compliance, access control, and deployment-appropriate safety layers. Review the upstream model card before downloading or deploying this derivative. The weights are provided as-is, without warranty.

Credits and license

• Base architecture: Qwen/Qwen3.8-27B
• Quantization source: AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16
• V100 runtime: 1CatAI/1Cat-vLLM

Apache-2.0, inherited from the source checkpoint and Qwen base model.