Granite-4.2-3B (FP8)
[!NOTE] This repository contains the FP8 version of Granite-4.2-3B.
Please refer to the original model's model card for additional details: https://huggingface.co/ibm-granite/granite-4.2-3b
[!NOTE] This repository contains the FP8 version of Granite-4.2-3B.
Please refer to the original model's model card for additional details: https://huggingface.co/ibm-granite/granite-4.2-3b
有文件？提交种子。
提交种子
LocalMaxxingvllm serve ibm-granite/granite-4.2-3b-fp8 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code在 LocalMaxxing 上查看测试
vllm serve ibm-granite/granite-4.2-3b-fp8 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code在 LocalMaxxing 上查看测试
vllm serve ibm-granite/granite-4.2-3b-fp8 --max-model-len 73728 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code在 LocalMaxxing 上查看测试
来自 LocalMaxxing 的 8 次社区测试（3 种配置）的中位数。
自己运行：配置指南