The model was quantized from Qwen/Qwen3.5-35B-A3B-FP8 using AMD-Quark. The weights are quantized to MXFP4 and activations are quantized to MXFP4.
Quantization scripts:
cd Quark/examples/torch/language_modeling/llm_ptq/
export exclude_layers="lm_head model.visual.* mtp.* *mlp.gate *shared_expert_gate* *.linear_attn.* *.self_attn.* *.shared_expert.*"
python3 quantize_quark.py --model_dir Qwen/Qwen3.5-35B-A3B-FP8 \
--quant_scheme mxfp4 \
--file2file_quantization \
--exclude_layers $exclude_layers \
--output_dir amd/Qwen3.5-35B-A3B-MXFP4
For further details or issues, please refer to the AMD-Quark documentation or contact the respective developers.
The model was evaluated on gsm8k benchmarks using the vllm framework.
| Benchmark | Qwen/Qwen3.5-35B-A3B-FP8 | amd/Qwen3.5-35B-A3B-MXFP4(this model) | Recovery |
| gsm8k (flexible-extract) | 89.39 | 93.25 | 104.32% |
The GSM8K results were obtained using the vLLM framework, based on the Docker image [rocm/vllm-dev:nightly_main_20260211], and vLLM is installed inside the container.
docker pull rocm/vllm-dev:nightly_main_20260211
lm_eval \
--model vllm \
--model_args pretrained=amd/Qwen3.5-35B-A3B-MXFP4,tensor_parallel_size=4,max_model_len=262144,gpu_memory_utilization=0.90,max_gen_toks=2048,trust_remote_code=True,reasoning_parser=qwen3 \
--tasks gsm8k --num_fewshot 5 \
--batch_size auto
Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
model.safetensors-00013-of-00014.safetensors 3.8 GBmodel.safetensors-00009-of-00014.safetensors 2.9 GBmodel.safetensors-00014-of-00014.safetensors 2.2 GBmodel.safetensors-00011-of-00014.safetensors 1.4 GBmodel.safetensors-00012-of-00014.safetensors 1.4 GBmodel.safetensors-00010-of-00014.safetensors 1.4 GBmodel.safetensors-00001-of-00014.safetensors 1.4 GBmodel.safetensors-00005-of-00014.safetensors 1.4 GBmodel.safetensors-00004-of-00014.safetensors 1.4 GBmodel.safetensors-00006-of-00014.safetensors 1.4 GBmodel.safetensors-00002-of-00014.safetensors 1.4 GBmodel.safetensors-00003-of-00014.safetensors 1.4 GBmodel.safetensors-00007-of-00014.safetensors 1.4 GBmodel.safetensors-00008-of-00014.safetensors 1.4 GB有文件?提交种子。
提交种子Qwen/Qwen3.5-35B-A3B-FP8