Qwen3.5-35B-A3B Uncensored (FP8_BLOCK)

A safetensors conversion and quantization of HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive (GGUF).

Model Details

| | |

|---|---|

| Architecture | Qwen3.5 MoE hybrid attention (30 GDN + 10 full standard attention layers) |

| Parameters | 35B-A3B |

| Base model |Qwen/Qwen3.5-35B-A3B |

| Source GGUF | HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive |

| Format | BF16 (Mixed/Compressed) |

| Quantization | FP8_BLOCK applied to Linear transformer layers. |

| Stripped layers | Multi-Token Prediction (MTP) due to original HF -> GGUF conversion. |

| Conversion type | Lossless GGUF to safetensors conversion + merge with base model vision layers + Block-wise quantization |

| Unquantized weights | Coming Soon |

---

Conversion Details

Converted using coming soon, which reverses transforms applied during HF -> GGUF conversion.

The vision encoder weights are copied directly from the official Qwen/Qwen3.5-35B-A3B base model, after confirming the vision encoder (mmproj) was not modified in the source GGUF.

Next, the linear weights of the transformer blocks were quantized to F8_E4M3 using llm-compressor.

Test Inference Details

• 1x A100 (80GB)
• Python 3.12
• vllm & transformers version:
• transformers 5.5.0
• vllm nightly (latest commit tested: 7b743ba)
• vLLM online serve flags:
• --quantization compressed-tensors
• --max-model-len 16384
• --gpu-memory-utilization 0.9140 with VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=1 environmental variable
• --limit-mm-per-prompt.image 4
• --enable-prefix-caching
• --enable-expert-parallel
• --reasoning-parser qwen3
• --default-chat-template-kwargs {"enable_thinking": false} disabled thinking/reasoning for vllm>=0.18.1
• Note: Used for batch image captioning tests.

---

Credits

• Base model: Qwen Team — Qwen3.5
• Ablation/uncensoring: HauhauCS
• Quantization tool: llm-compressor