Granite-4.2-8B (NVFP4)
[!NOTE] This repository contains the NVFP4 version of Granite-4.2-8B.
Please refer to the original model's model card for additional details: https://huggingface.co/ibm-granite/granite-4.2-8b
[!NOTE] This repository contains the NVFP4 version of Granite-4.2-8B.
Please refer to the original model's model card for additional details: https://huggingface.co/ibm-granite/granite-4.2-8b
Have the files? Submit a torrent.
Submit a torrent →Have the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo ibm-granite/granite-4.2-8b-nvfp4 ./model-folder
LocalMaxxingvllm serve ibm-granite/granite-4.2-8b-nvfp4 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-codeView run on LocalMaxxing
vllm serve ibm-granite/granite-4.2-8b-nvfp4 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code --kv-cache-dtype int4_per_token_head --attention-backend TRITON_ATTNView run on LocalMaxxing
vllm serve ibm-granite/granite-4.2-8b-nvfp4 --max-model-len 24576 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-codeView run on LocalMaxxing
Median of 16 community runs on 3 setups, from LocalMaxxing.
Run it yourself: setup guide