Beinsezii/Mistral-Small-4-119B-2603-GGUF-HALO

🤗 Hugging Face sourceapache-2.0119B activated99 GBGGUF✓ 2 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo Beinsezii/Mistral-Small-4-119B-2603-GGUF-HALO ./model-folder
Needs a seeder →

Quant optimized for quality / speed on a Strix Halo 128GiB system. Possibly also beneficial on DGX Spark and similar systems.

The TL;DR is this quant achieves both superior quality and speed compared to homogenous Q6_K.

See the GLM version for more details on theory and comparisons.