Granite-4.2-3B (FP8)
[!NOTE] This repository contains the FP8 version of Granite-4.2-3B.
Please refer to the original model's model card for additional details: https://huggingface.co/ibm-granite/granite-4.2-3b
[!NOTE] This repository contains the FP8 version of Granite-4.2-3B.
Please refer to the original model's model card for additional details: https://huggingface.co/ibm-granite/granite-4.2-3b
Have the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo ibm-granite/granite-4.2-3b-fp8 ./model-folder
LocalMaxxingvllm serve ibm-granite/granite-4.2-3b-fp8 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-codeView run on LocalMaxxing
vllm serve ibm-granite/granite-4.2-3b-fp8 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-codeView run on LocalMaxxing
vllm serve ibm-granite/granite-4.2-3b-fp8 --max-model-len 73728 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-codeView run on LocalMaxxing
Median of 8 community runs on 3 setups, from LocalMaxxing.
Run it yourself: setup guide