Granite-4.2-30B (NVFP4)
[!NOTE] This repository contains the NVFP4 version of Granite-4.2-30B.
Please refer to the original model's model card for additional details: https://huggingface.co/ibm-granite/granite-4.2-30b
[!NOTE] This repository contains the NVFP4 version of Granite-4.2-30B.
Please refer to the original model's model card for additional details: https://huggingface.co/ibm-granite/granite-4.2-30b
有文件？提交种子。
提交种子
LocalMaxxingvllm serve ibm-granite/granite-4.2-30b-nvfp4 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code在 LocalMaxxing 上查看测试
vllm serve ibm-granite/granite-4.2-30b-nvfp4 --max-model-len 57344 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code --kv-cache-dtype int4_per_token_head --attention-backend TRITON_ATTN在 LocalMaxxing 上查看测试
来自 LocalMaxxing 的 8 次社区测试（2 种配置）的中位数。
自己运行：配置指南