Qwen3.5-122B-A10B-heretic-hybrid-int4fp8 (+MTP)

Uncensored Qwen3.5-122B-A10B (MoE, 122B total / ~10B active) quantized as a hybrid INT4 (MoE experts)

• FP8 (shared-expert / dense) checkpoint with working MTP speculative decoding, tuned to run on a single NVIDIA DGX Spark (GB10, 121 GB unified).

• Base (abliterated): trohrbaugh/Qwen3.5-122B-A10B-heretic (KL≈0.09)
• Quant: Intel AutoRound W4A16, group 128, model_free RTN; shared-expert → FP8 E4M3 (block-128) from Qwen/Qwen3.5-122B-A10B-FP8
• MTP: 785 BF16 heads (from OptimizeLLM/Qwen3.5-122B-A10B-heretic-MTP-NVFP4)
• Size: ~67 GB on disk · Context: 256k · Vision: tower kept BF16 (multimodal preserved)
• Uncensored: abliterated base; it does not refuse.

Measured (GB10, single stream, max-num-seqs 1)

Serve (vLLM)

Needs a vLLM built for sm_121 with the hybrid-FP8 dispatch patch — see albond/DGX_Spark_Qwen3.5-122B-A10B-AR-INT4 (image vllm-qwen35-v2). Also loads on stock vLLM as plain INT4 (slower, no FP8 shared-expert).

vllm serve GaelicThunder/Qwen3.5-122B-A10B-heretic-hybrid-int4fp8 \
  --max-model-len 262144 --gpu-memory-utilization 0.75 --trust-remote-code \
  --load-format fastsafetensors \
  --enable-chunked-prefill --max-num-batched-tokens 8192 --max-num-seqs 1 \
  --reasoning-parser qwen3 --attention-backend FLASHINFER \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}' \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml

⚠ Critical config note (MTP)

config.json sets quantization_config.block_name_to_quantize = "model.language_model.layers". This is required: it scopes quantization to the main transformer blocks so the MTP module (mtp.*) and vision tower stay unquantized. Without it, vLLM's qwen3_5_mtp loader raises KeyError: 'layers.0.mlp.experts.w2_weight'.

Build

Full recipe + caveats + scripts: https://github.com/GaelicThunder/heretic-122b-dgx-spark

License & responsible use

Apache-2.0 (inherited from Qwen/Qwen3.5-122B-A10B — verify). Abliteration by trohrbaugh. This model is uncensored and will comply with requests a safety-tuned model would refuse. You are responsible for lawful, ethical use.