This model was built with GLM-5.3 model by applying AMD-Quark for MXFP4 quantization.
The model was quantized from zai-org/GLM-5.3 using AMD-Quark. The weights and activations are quantized to MXFP4.
Quantization scripts:
cd Quark/examples/torch/language_modeling/llm_ptq/
python quantize_quark.py \
--model_dir zai-org/GLM-5.3 \
--output_dir amd/GLM-5.3-Quark-MXFP4 \
--quant_scheme mxfp4 \
--exclude_layers \
"*self_attn*" \
"*lm_head" \
"*embed_tokens" \
"*eh_proj" \
"*mlp.gate" \
--file2file_quantization \
--keep_excluded_layers_as_original_model_state
The model was evaluated using vLLM, EvalScope, and lm-evaluation-harness.
| Benchmark | zai-org/GLM-5.3 | amd/GLM-5.3-Quark-MXFP4-AttnFP8 (this model) | Recovery |
|---|---|---|---|
| GSM8K (flexible-extract) | 91.96 | 91.43 | 99.42% |
| GPQA Diamond (mean@8) | 92.36 | 91.54 | 99.11% |
| AIME26 (mean@16) | 98.12 | 98.96 | 100.86% |
| MMLU Pro (mean@1) | 88.19 | 87.70 | 99.44% |
| SciCode, subproblem (mean@8) | 56.14 | 56.60 | 100.82% |
| AA-LCR (mean@16) | 72.44 | 71.69 | 98.96% |
| IFBench (mean@5) | 67.33 | 66.67 | 99.02% |
Baseline is zai-org/GLM-5.3 (native FP8). MMLU Pro is used in place of MMMU Pro as this model has no vision encoder.
Note: Results may vary from run to run.
The following results were obtained using vLLM, EvalScope 1.14.1, and lm-evaluation-harness.
We launch the vLLM server following vLLM Recipes using container image vllm/vllm-openai-rocm:v0.31.0.
VLLM_ROCM_USE_AITER=1 \
VLLM_ROCM_USE_AITER_FP8BMM=0 \
VLLM_ROCM_USE_AITER_FP4BMM=0 \
vllm serve amd/GLM-5.3-Quark-MXFP4-AttnFP8 \
--served-model-name GLM-5.3-MXFP4 \
--host 0.0.0.0 \
--port 8000 \
--tensor-parallel-size 8 \
--kv-cache-dtype fp8_e4m3 \
--trust-remote-code \
--max-model-len 400000 \
--max-num-seqs 128 \
--reasoning-parser glm47 \
--tool-call-parser glm47 \
--enable-auto-tool-choice
Note: GPQA Diamond, AIME26, and MMLU Pro were evaluated with
--max-model-len 327680; SciCode requires--max-model-len 400000.
GSM8K
pip install lm-eval[api]
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_FP8BMM=0
export VLLM_ROCM_USE_AITER_FP4BMM=0
lm_eval --model vllm \
--model_args "pretrained=amd/GLM-5.3-Quark-MXFP4,tensor_parallel_size=4,dtype=auto,quantization=quark,max_model_len=32768,trust_remote_code=True,max_num_seqs=32" \
--tasks gsm8k \
--batch_size auto
GPQA Diamond
evalscope eval \
--model GLM-5.3-MXFP4 \
--api-url http://127.0.0.1:8000/v1/chat/completions \
--datasets gpqa_diamond \
--eval-batch-size 128 \
--repeats 8 \
--generation-config '{"temperature":1.0,"top_p":0.95,"max_tokens":320000,"timeout":43200}' \
--work-dir logs/gpqa_diamond
AIME26
evalscope eval \
--model GLM-5.3-MXFP4 \
--api-url http://127.0.0.1:8000/v1/chat/completions \
--datasets aime26 \
--eval-batch-size 128 \
--repeats 16 \
--generation-config '{"temperature":1.0,"top_p":0.95,"max_tokens":320000,"timeout":7200}' \
--work-dir logs/aime26
MMLU Pro
evalscope eval \
--model GLM-5.3-MXFP4 \
--api-url http://127.0.0.1:8000/v1/chat/completions \
--datasets mmlu_pro \
--eval-batch-size 128 \
--repeats 1 \
--generation-config '{"temperature":1.0,"top_p":0.95,"max_tokens":320000,"timeout":43200}' \
--work-dir logs/mmlu_pro
SciCode
evalscope eval \
--model GLM-5.3-MXFP4 \
--api-url http://127.0.0.1:8000/v1/chat/completions \
--datasets scicode \
--eval-batch-size 128 \
--repeats 8 \
--generation-config '{"temperature":1.0,"top_p":0.95,"max_tokens":320000,"timeout":43200,"extra_body":{"chat_template_kwargs":{"clear_thinking":true}}}' \
--dataset-args '{"scicode":{"review_timeout":1800,"extra_params":{"provide_background":true}}}' \
--sandbox '{"enabled":true,"engine":"docker","pool_size":16}' \
--work-dir logs/scicode
AA-LCR (v1.1)
Judge configuration (aa-lcr-judge.json; replace <YOUR_SUBSCRIPTION_KEY>):
{
"strategy": "llm",
"models": [
{
"model_id": "gpt-5.6-luna",
"api_url": "https://llm-api.amd.com/openai/responses",
"api_key": "EMPTY",
"eval_type": "openai_responses_api",
"generation_config": {
"reasoning_effort": "medium",
"max_tokens": 4096,
"timeout": 7200,
"extra_query": {
"api-version": "2025-04-01-preview"
},
"extra_headers": {
"Ocp-Apim-Subscription-Key": "<YOUR_SUBSCRIPTION_KEY>"
}
}
}
]
}
evalscope eval \
--model GLM-5.3-MXFP4 \
--api-url http://127.0.0.1:8000/v1/chat/completions \
--datasets aa_lcr \
--dataset-hub huggingface \
--eval-batch-size 64 \
--repeats 16 \
--generation-config '{"temperature":1.0,"top_p":0.95,"max_tokens":16384,"timeout":43200}' \
--dataset-args '{"aa_lcr":{"hf_dataset_revision":"9a77ef56b717057ade24ceab4d273712a0b4f19e"}}' \
--judge "$(cat aa-lcr-judge.json)" \
--work-dir logs/aa_lcr
IFBench
evalscope eval \
--model GLM-5.3-MXFP4 \
--api-url http://127.0.0.1:8000/v1/chat/completions \
--datasets ifbench \
--eval-batch-size 128 \
--repeats 5 \
--generation-config '{"temperature":1.0,"top_p":0.95,"max_tokens":320000,"timeout":43200}' \
--dataset-args '{"ifbench":{"primary_metric":"prompt_level_loose"}}' \
--work-dir logs/ifbench
Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
model-00001-of-00141.safetensors 4.8 GBmodel-00020-of-00141.safetensors 3.0 GBmodel-00136-of-00141.safetensors 3.0 GBmodel-00138-of-00141.safetensors 2.9 GBmodel-00011-of-00141.safetensors 2.9 GBmodel-00057-of-00141.safetensors 2.9 GBmodel-00033-of-00141.safetensors 2.9 GBmodel-00114-of-00141.safetensors 2.9 GBmodel-00090-of-00141.safetensors 2.9 GBmodel-00066-of-00141.safetensors 2.9 GBmodel-00018-of-00141.safetensors 2.9 GBmodel-00042-of-00141.safetensors 2.9 GBmodel-00123-of-00141.safetensors 2.9 GBmodel-00003-of-00141.safetensors 2.9 GBmodel-00025-of-00141.safetensors 2.9 GBmodel-00049-of-00141.safetensors 2.9 GBmodel-00082-of-00141.safetensors 2.9 GBmodel-00106-of-00141.safetensors 2.9 GBmodel-00130-of-00141.safetensors 2.9 GBmodel-00073-of-00141.safetensors 2.9 GBmodel-00097-of-00141.safetensors 2.9 GBmodel-00101-of-00141.safetensors 2.9 GBmodel-00070-of-00141.safetensors 2.9 GBmodel-00035-of-00141.safetensors 2.9 GBmodel-00116-of-00141.safetensors 2.9 GB 2.9 GBmodel-00046-of-00141.safetensorsmodel-00127-of-00141.safetensors 2.9 GBmodel-00092-of-00141.safetensors 2.9 GBmodel-00022-of-00141.safetensors 2.9 GBmodel-00081-of-00141.safetensors 2.9 GBmodel-00068-of-00141.safetensors 2.9 GBmodel-00103-of-00141.safetensors 2.9 GBmodel-00009-of-00141.safetensors 2.9 GBmodel-00044-of-00141.safetensors 2.9 GBmodel-00125-of-00141.safetensors 2.9 GBmodel-00055-of-00141.safetensors 2.9 GBmodel-00007-of-00141.safetensors 2.9 GBmodel-00031-of-00141.safetensors 2.9 GBmodel-00112-of-00141.safetensors 2.9 GBmodel-00077-of-00141.safetensors 2.9 GBmodel-00088-of-00141.safetensors 2.9 GBmodel-00053-of-00141.safetensors 2.9 GBmodel-00134-of-00141.safetensors 2.9 GBmodel-00064-of-00141.safetensors 2.9 GBmodel-00005-of-00141.safetensors 2.9 GBmodel-00029-of-00141.safetensors 2.9 GBmodel-00110-of-00141.safetensors 2.9 GBmodel-00075-of-00141.safetensors 2.9 GBmodel-00016-of-00141.safetensors 2.9 GBmodel-00012-of-00141.safetensors 2.9 GBmodel-00027-of-00141.safetensors 2.9 GBmodel-00036-of-00141.safetensors 2.9 GBmodel-00047-of-00141.safetensors 2.9 GBmodel-00051-of-00141.safetensors 2.9 GBmodel-00062-of-00141.safetensors 2.9 GBmodel-00084-of-00141.safetensors 2.9 GBmodel-00086-of-00141.safetensors 2.9 GBmodel-00093-of-00141.safetensors 2.9 GBmodel-00095-of-00141.safetensors 2.9 GBmodel-00108-of-00141.safetensors 2.9 GBmodel-00117-of-00141.safetensors 2.9 GBmodel-00132-of-00141.safetensors 2.9 GBmodel-00014-of-00141.safetensors 2.9 GBmodel-00071-of-00141.safetensors 2.9 GBmodel-00128-of-00141.safetensors 2.9 GBmodel-00038-of-00141.safetensors 2.9 GBmodel-00119-of-00141.safetensors 2.9 GBmodel-00079-of-00141.safetensors 2.9 GBmodel-00099-of-00141.safetensors 2.9 GBmodel-00140-of-00141.safetensors 2.9 GBmodel-00040-of-00141.safetensors 2.9 GBmodel-00121-of-00141.safetensors 2.9 GBmodel-00060-of-00141.safetensors 2.9 GBmodel-00105-of-00141.safetensors 2.9 GBmodel-00024-of-00141.safetensors 2.9 GBmodel-00058-of-00141.safetensors 2.9 GBmodel-00059-of-00141.safetensors 2.9 GBmodel-00023-of-00141.safetensors 2.9 GBmodel-00104-of-00141.safetensors 2.9 GBmodel-00072-of-00141.safetensors 2.8 GBmodel-00002-of-00141.safetensors 2.8 GBmodel-00013-of-00141.safetensors 2.8 GBmodel-00026-of-00141.safetensors 2.8 GBmodel-00037-of-00141.safetensors 2.8 GBmodel-00048-of-00141.safetensors 2.8 GBmodel-00050-of-00141.safetensors 2.8 GBmodel-00061-of-00141.safetensors 2.8 GBmodel-00083-of-00141.safetensors 2.8 GBmodel-00085-of-00141.safetensors 2.8 GBmodel-00094-of-00141.safetensors 2.8 GBmodel-00096-of-00141.safetensors 2.8 GBmodel-00107-of-00141.safetensors 2.8 GBmodel-00118-of-00141.safetensors 2.8 GBmodel-00129-of-00141.safetensors 2.8 GBmodel-00131-of-00141.safetensors 2.8 GBmodel-00015-of-00141.safetensors 2.8 GBmodel-00074-of-00141.safetensors 2.8 GBmodel-00028-of-00141.safetensors 2.8 GBmodel-00109-of-00141.safetensors 2.8 GBmodel-00004-of-00141.safetensors 2.8 GBmodel-00063-of-00141.safetensors 2.8 GBmodel-00098-of-00141.safetensors 2.8 GBmodel-00052-of-00141.safetensors 2.8 GBmodel-00133-of-00141.safetensors 2.8 GBmodel-00087-of-00141.safetensors 2.8 GBmodel-00041-of-00141.safetensors 2.8 GBmodel-00122-of-00141.safetensors 2.8 GBmodel-00017-of-00141.safetensors 2.8 GBmodel-00076-of-00141.safetensors 2.8 GBmodel-00030-of-00141.safetensors 2.8 GBmodel-00111-of-00141.safetensors 2.8 GBmodel-00006-of-00141.safetensors 2.8 GBmodel-00065-of-00141.safetensors 2.8 GBmodel-00054-of-00141.safetensors 2.8 GBmodel-00135-of-00141.safetensors 2.8 GBmodel-00089-of-00141.safetensors 2.8 GBmodel-00019-of-00141.safetensors 2.8 GBmodel-00043-of-00141.safetensors 2.8 GBmodel-00124-of-00141.safetensors 2.8 GBmodel-00078-of-00141.safetensors 2.8 GBmodel-00032-of-00141.safetensors 2.8 GBmodel-00113-of-00141.safetensors 2.8 GBmodel-00008-of-00141.safetensors 2.8 GBmodel-00067-of-00141.safetensors 2.8 GBmodel-00137-of-00141.safetensors 2.8 GBmodel-00102-of-00141.safetensors 2.8 GBmodel-00021-of-00141.safetensors 2.8 GBmodel-00056-of-00141.safetensors 2.8 GBmodel-00091-of-00141.safetensors 2.8 GBmodel-00045-of-00141.safetensors 2.8 GBmodel-00126-of-00141.safetensors 2.8 GBmodel-00115-of-00141.safetensors 2.8 GBmodel-00034-of-00141.safetensors 2.8 GBmodel-00010-of-00141.safetensors 2.8 GBmodel-00069-of-00141.safetensors 2.8 GBmodel-00120-of-00141.safetensors 2.8 GBmodel-00039-of-00141.safetensors 2.8 GBmodel-00100-of-00141.safetensors 2.8 GBmodel-00080-of-00141.safetensors 2.8 GBmodel-00139-of-00141.safetensors 2.8 GBmodel-00141-of-00141.safetensors 2.6 GBzai-org/GLM-5.3