This repository contains unofficial GGUF Quantizations of Grok-1, compatible with llama.cpp as of PR- Add grok-1 support #6204.
The splits have been updated to utilize the improvements from PR: llama_model_loader: support multiple split/shard GGUFs. As a result, manual merging with gguf-split is no longer required.
With this, there is no need to merge the split files before use. Just download all splits and run llama.cpp with the first split like you would previously. It'll detect the other splits and load them as well.
Thanks to a new PR common: llama_load_model_from_url split support #6192 from phymbert it's now possible load model splits from url.
That means this downloads and runs the model:
server \
--hf-repo Arki05/Grok-1-GGUF \
--hf-file grok-1-IQ3_XS-split-00001-of-00009.gguf \
--model models/grok-1-IQ3_XS-split-00001-of-00009.gguf \
-ngl 999
And that is very cool (@phymbert)
The following Quantizations are currently available for download:
| Quant | Split Files | Size |
|---|---|---|
Q2_K |
1-of-9, 2-of-9, 3-of-9, 4-of-9, 5-of-9, 6-of-9, 7-of-9, 8-of-9, 9-of-9 | 112.4 GB |
IQ3_XS |
1-of-9, 2-of-9, 3-of-9, 4-of-9, 5-of-9, 6-of-9, 7-of-9, 8-of-9, 9-of-9 | 125.4 GB |
Q4_K |
1-of-9, 2-of-9, 3-of-9, 4-of-9, 5-of-9, 6-of-9, 7-of-9, 8-of-9, 9-of-9 | 186.0 GB |
Q6_K |
1-of-9, 2-of-9, 3-of-9, 4-of-9, 5-of-9, 6-of-9, 7-of-9, 8-of-9, 9-of-9 | 259.8 GB |
I would recommend the IQ3_XS version for now.
More Quantizations will be uploaded soon. All current Quants are created without any importance matrix.
IQ1_S/grok-1-IQ1_S-00001-of-00009.gguf 9.2 GBIQ1_S/grok-1-IQ1_S-00002-of-00009.gguf 7.5 GBIQ1_S/grok-1-IQ1_S-00005-of-00009.gguf 7.2 GBIQ1_S/grok-1-IQ1_S-00006-of-00009.gguf 7.2 GBIQ1_S/grok-1-IQ1_S-00003-of-00009.gguf 7.2 GBIQ1_S/grok-1-IQ1_S-00007-of-00009.gguf 7.1 GBIQ1_S/grok-1-IQ1_S-00008-of-00009.gguf 7.0 GBIQ1_S/grok-1-IQ1_S-00004-of-00009.gguf 7.0 GBIQ1_S/grok-1-IQ1_S-00009-of-00009.gguf 5.7 GBIQ1_M/grok-1-IQ1_M-00001-of-00009.gguf 11 GBIQ1_M/grok-1-IQ1_M-00005-of-00009.gguf 8.7 GBIQ1_M/grok-1-IQ1_M-00004-of-00009.gguf 8.5 GBIQ1_M/grok-1-IQ1_M-00006-of-00009.gguf 8.4 GBIQ1_M/grok-1-IQ1_M-00003-of-00009.gguf 8.4 GBIQ1_M/grok-1-IQ1_M-00007-of-00009.gguf 8.4 GBIQ1_M/grok-1-IQ1_M-00008-of-00009.gguf 8.4 GBIQ1_M/grok-1-IQ1_M-00002-of-00009.gguf 8.4 GBIQ1_M/grok-1-IQ1_M-00009-of-00009.gguf 2.2 GBgrok-1-Q2_K-split-00001-of-00009.gguf 15 GBgrok-1-Q2_K-split-00004-of-00009.gguf 14 GBgrok-1-Q2_K-split-00008-of-00009.gguf 14 GBgrok-1-Q2_K-split-00005-of-00009.gguf 14 GBgrok-1-Q2_K-split-00006-of-00009.gguf 14 GBgrok-1-Q2_K-split-00003-of-00009.gguf 14 GBgrok-1-Q2_K-split-00007-of-00009.gguf 14 GBgrok-1-Q2_K-split-00002-of-00009.gguf 14 GBgrok-1-Q2_K-split-00009-of-00009.gguf 3.6 GBgrok-1-IQ3_XS-split-00001-of-00009.gguf 18 GBgrok-1-IQ3_XS-split-00008-of-00009.gguf 16 GBgrok-1-IQ3_XS-split-00004-of-00009.gguf 16 GBgrok-1-IQ3_XS-split-00005-of-00009.gguf 15 GBgrok-1-IQ3_XS-split-00002-of-00009.gguf 15 GBgrok-1-IQ3_XS-split-00006-of-00009.gguf 15 GBgrok-1-IQ3_XS-split-00003-of-00009.gguf 15 GBgrok-1-IQ3_XS-split-00007-of-00009.gguf 15 GBgrok-1-IQ3_XS-split-00009-of-00009.gguf 4.2 GBIQ3_S/grok-1-IQ3_S-split-00001-of-00009.gguf 16 GBIQ3_S/grok-1-IQ3_S-split-00005-of-00009.gguf 16 GBIQ3_S/grok-1-IQ3_S-split-00006-of-00009.gguf 16 GBIQ3_S/grok-1-IQ3_S-split-00002-of-00009.gguf 16 GBIQ3_S/grok-1-IQ3_S-split-00003-of-00009.gguf 16 GBIQ3_S/grok-1-IQ3_S-split-00007-of-00009.gguf 15 GBIQ3_S/grok-1-IQ3_S-split-00008-of-00009.gguf 15 GBIQ3_S/grok-1-IQ3_S-split-00004-of-00009.gguf 15 GBIQ3_S/grok-1-IQ3_S-split-00009-of-00009.gguf 12 GBgrok-1-Q4_K-split-00001-of-00009.gguf 26 GBgrok-1-Q4_K-split-00008-of-00009.gguf 25 GBgrok-1-Q4_K-split-00005-of-00009.gguf 23 GBgrok-1-Q4_K-split-00004-of-00009.gguf 23 GBgrok-1-Q4_K-split-00003-of-00009.gguf 23 GBgrok-1-Q4_K-split-00006-of-00009.gguf 22 GBgrok-1-Q4_K-split-00007-of-00009.gguf 22 GBgrok-1-Q4_K-split-00002-of-00009.gguf 22 GBgrok-1-Q4_K-split-00009-of-00009.gguf 6.4 GBgrok-1-Q6_K-split-00001-of-00009.gguf 33 GBgrok-1-Q6_K-split-00004-of-00009.gguf 32 GBgrok-1-Q6_K-split-00008-of-00009.gguf 32 GBgrok-1-Q6_K-split-00005-of-00009.gguf 31 GBgrok-1-Q6_K-split-00003-of-00009.gguf 31 GBgrok-1-Q6_K-split-00006-of-00009.gguf 31 GBgrok-1-Q6_K-split-00007-of-00009.gguf 31 GBgrok-1-Q6_K-split-00002-of-00009.gguf 31 GBgrok-1-Q6_K-split-00009-of-00009.gguf 8.1 GBIQ2_XXS/grok-1-IQ2_XXS-00001-of-00009.gguf 11 GBIQ2_XXS/grok-1-IQ2_XXS-00002-of-00009.gguf 9.6 GBIQ2_XXS/grok-1-IQ2_XXS-00005-of-00009.gguf 9.5 GBIQ2_XXS/grok-1-IQ2_XXS-00006-of-00009.gguf 9.5 GBIQ2_XXS/grok-1-IQ2_XXS-00003-of-00009.gguf 9.4 GBIQ2_XXS/grok-1-IQ2_XXS-00007-of-00009.gguf 9.3 GBIQ2_XXS/grok-1-IQ2_XXS-00008-of-00009.gguf 9.2 GBIQ2_XXS/grok-1-IQ2_XXS-00004-of-00009.gguf 9.1 GBIQ2_XXS/grok-1-IQ2_XXS-00009-of-00009.gguf 7.5 GBIQ2_XS/grok-1-IQ2_XS-00001-of-00009.gguf 11 GBIQ2_XS/grok-1-IQ2_XS-00002-of-00009.gguf 11 GBIQ2_XS/grok-1-IQ2_XS-00005-of-00009.gguf 11 GBIQ2_XS/grok-1-IQ2_XS-00006-of-00009.gguf 11 GBIQ2_XS/grok-1-IQ2_XS-00003-of-00009.gguf 11 GBIQ2_XS/grok-1-IQ2_XS-00007-of-00009.gguf 10 GBIQ2_XS/grok-1-IQ2_XS-00008-of-00009.gguf 10 GBIQ2_XS/grok-1-IQ2_XS-00004-of-00009.gguf 10 GBIQ2_XS/grok-1-IQ2_XS-00009-of-00009.gguf 8.4 GBIQ2_S/grok-1-IQ2_S-00001-of-00009.gguf 13 GB 11 GBIQ3_XXS/grok-1-IQ3_XXS-00001-of-00009.gguf 14 GBIQ3_XXS/grok-1-IQ3_XXS-00005-of-00009.gguf 14 GBIQ3_XXS/grok-1-IQ3_XXS-00006-of-00009.gguf 14 GBIQ3_XXS/grok-1-IQ3_XXS-00002-of-00009.gguf 14 GBIQ3_XXS/grok-1-IQ3_XXS-00003-of-00009.gguf 14 GBIQ3_XXS/grok-1-IQ3_XXS-00007-of-00009.gguf 14 GBIQ3_XXS/grok-1-IQ3_XXS-00008-of-00009.gguf 14 GBIQ3_XXS/grok-1-IQ3_XXS-00004-of-00009.gguf 13 GBIQ3_XXS/grok-1-IQ3_XXS-00009-of-00009.gguf 11 GBIQ3_XS/grok-1-IQ3_XS-00001-of-00009.gguf 16 GBIQ3_XS/grok-1-IQ3_XS-00002-of-00009.gguf 15 GBIQ3_XS/grok-1-IQ3_XS-00006-of-00009.gguf 15 GBIQ3_XS/grok-1-IQ3_XS-00005-of-00009.gguf 15 GBIQ3_XS/grok-1-IQ3_XS-00008-of-00009.gguf 15 GBIQ3_XS/grok-1-IQ3_XS-00003-of-00009.gguf 14 GBIQ3_XS/grok-1-IQ3_XS-00007-of-00009.gguf 14 GBIQ3_XS/grok-1-IQ3_XS-00004-of-00009.gguf 14 GBIQ3_XS/grok-1-IQ3_XS-00009-of-00009.gguf 12 GBQ3_K_S/grok-1-Q3_K_S-00001-of-00009.gguf 16 GB 16 GBIQ4_XS/grok-1-IQ4_XS-00001-of-00009.gguf 20 GBIQ4_XS/grok-1-IQ4_XS-00005-of-00009.gguf 19 GBIQ4_XS/grok-1-IQ4_XS-00006-of-00009.gguf 19 GBIQ4_XS/grok-1-IQ4_XS-00002-of-00009.gguf 19 GBIQ4_XS/grok-1-IQ4_XS-00003-of-00009.gguf 19 GBIQ4_XS/grok-1-IQ4_XS-00007-of-00009.gguf 19 GBIQ4_XS/grok-1-IQ4_XS-00008-of-00009.gguf 19 GBIQ4_XS/grok-1-IQ4_XS-00004-of-00009.gguf 19 GBIQ4_XS/grok-1-IQ4_XS-00009-of-00009.gguf 15 GBIQ4_NL/grok-1-IQ4_NL-00001-of-00009.gguf 21 GBIQ4_NL/grok-1-IQ4_NL-00005-of-00009.gguf 20 GBIQ4_NL/grok-1-IQ4_NL-00006-of-00009.gguf 20 GBIQ4_NL/grok-1-IQ4_NL-00002-of-00009.gguf 20 GBIQ4_NL/grok-1-IQ4_NL-00003-of-00009.gguf 20 GBIQ4_NL/grok-1-IQ4_NL-00007-of-00009.gguf 20 GBIQ4_NL/grok-1-IQ4_NL-00008-of-00009.gguf 20 GBIQ4_NL/grok-1-IQ4_NL-00004-of-00009.gguf 20 GBIQ4_NL/grok-1-IQ4_NL-00009-of-00009.gguf 16 GBQ4_K/grok-1-Q4_K-00001-of-00009.gguf 24 GBQ4_K/grok-1-Q4_K-00006-of-00009.gguf 22 GBQ5_K/grok-1-Q5_K-00001-of-00009.gguf 27 GBQ5_K/grok-1-Q5_K-00006-of-00009.gguf 26 GBQ5_K/grok-1-Q5_K-00002-of-00009.gguf 26 GBQ5_K/grok-1-Q5_K-00005-of-00009.gguf 25 GBQ5_K/grok-1-Q5_K-00003-of-00009.gguf 25 GBQ5_K/grok-1-Q5_K-00008-of-00009.gguf 25 GBQ5_K/grok-1-Q5_K-00007-of-00009.gguf 25 GBQ5_K/grok-1-Q5_K-00004-of-00009.gguf 25 GBQ5_K/grok-1-Q5_K-00009-of-00009.gguf 21 GBQ6_K/grok-1-Q6_K-00001-of-00009.gguf 30 GBQ6_K/grok-1-Q6_K-00005-of-00009.gguf 30 GBQ6_K/grok-1-Q6_K-00006-of-00009.gguf 30 GBQ6_K/grok-1-Q6_K-00002-of-00009.gguf 30 GBQ6_K/grok-1-Q6_K-00003-of-00009.gguf 30 GBQ6_K/grok-1-Q6_K-00007-of-00009.gguf 29 GBQ6_K/grok-1-Q6_K-00008-of-00009.gguf 29 GBQ6_K/grok-1-Q6_K-00004-of-00009.gguf 29 GBQ6_K/grok-1-Q6_K-00009-of-00009.gguf 24 GBQ8_0/grok-1-Q8_0-00001-of-00009.gguf 39 GBQ8_0/grok-1-Q8_0-00005-of-00009.gguf 39 GBQ8_0/grok-1-Q8_0-00006-of-00009.gguf 39 GBQ8_0/grok-1-Q8_0-00002-of-00009.gguf 39 GBQ8_0/grok-1-Q8_0-00003-of-00009.gguf 38 GBQ8_0/grok-1-Q8_0-00007-of-00009.gguf 38 GBQ8_0/grok-1-Q8_0-00008-of-00009.gguf 37 GBQ8_0/grok-1-Q8_0-00004-of-00009.gguf 37 GBQ8_0/grok-1-Q8_0-00009-of-00009.gguf 31 GBIQ2_S/grok-1-IQ2_S-00002-of-00009.ggufIQ2_S/grok-1-IQ2_S-00005-of-00009.gguf 11 GBIQ2_S/grok-1-IQ2_S-00006-of-00009.gguf 11 GBIQ2_S/grok-1-IQ2_S-00003-of-00009.gguf 11 GBIQ2_S/grok-1-IQ2_S-00007-of-00009.gguf 10 GBIQ2_S/grok-1-IQ2_S-00008-of-00009.gguf 10 GBIQ2_S/grok-1-IQ2_S-00004-of-00009.gguf 10 GBIQ2_S/grok-1-IQ2_S-00009-of-00009.gguf 8.4 GBIQ2_M/grok-1-IQ2_M-00001-of-00009.gguf 13 GBIQ2_M/grok-1-IQ2_M-00002-of-00009.gguf 12 GBIQ2_M/grok-1-IQ2_M-00005-of-00009.gguf 12 GBIQ2_M/grok-1-IQ2_M-00006-of-00009.gguf 12 GBIQ2_M/grok-1-IQ2_M-00003-of-00009.gguf 12 GBIQ2_M/grok-1-IQ2_M-00007-of-00009.gguf 12 GBIQ2_M/grok-1-IQ2_M-00008-of-00009.gguf 11 GBIQ2_M/grok-1-IQ2_M-00004-of-00009.gguf 11 GBIQ2_M/grok-1-IQ2_M-00009-of-00009.gguf 9.3 GBQ2_K_S/grok-1-Q2_K_S-00001-of-00009.gguf 15 GBQ2_K_S/grok-1-Q2_K_S-00002-of-00009.gguf 12 GBQ2_K_S/grok-1-Q2_K_S-00005-of-00009.gguf 12 GBQ2_K_S/grok-1-Q2_K_S-00006-of-00009.gguf 12 GBQ2_K_S/grok-1-Q2_K_S-00003-of-00009.gguf 12 GBQ2_K_S/grok-1-Q2_K_S-00007-of-00009.gguf 12 GBQ2_K_S/grok-1-Q2_K_S-00008-of-00009.gguf 12 GBQ2_K_S/grok-1-Q2_K_S-00004-of-00009.gguf 11 GBQ2_K_S/grok-1-Q2_K_S-00009-of-00009.gguf 9.5 GBQ2_K/grok-1-Q2_K-00001-of-00009.gguf 14 GBQ2_K/grok-1-Q2_K-00002-of-00009.gguf 13 GBQ2_K/grok-1-Q2_K-00006-of-00009.gguf 13 GBQ2_K/grok-1-Q2_K-00005-of-00009.gguf 13 GBQ2_K/grok-1-Q2_K-00003-of-00009.gguf 13 GBQ2_K/grok-1-Q2_K-00007-of-00009.gguf 13 GBQ2_K/grok-1-Q2_K-00008-of-00009.gguf 13 GBQ2_K/grok-1-Q2_K-00004-of-00009.gguf 13 GBQ2_K/grok-1-Q2_K-00009-of-00009.gguf 11 GBQ3_K_S/grok-1-Q3_K_S-00005-of-00009.ggufQ3_K_S/grok-1-Q3_K_S-00006-of-00009.gguf 16 GBQ3_K_S/grok-1-Q3_K_S-00002-of-00009.gguf 16 GBQ3_K_S/grok-1-Q3_K_S-00003-of-00009.gguf 16 GBQ3_K_S/grok-1-Q3_K_S-00007-of-00009.gguf 15 GBQ3_K_S/grok-1-Q3_K_S-00008-of-00009.gguf 15 GBQ3_K_S/grok-1-Q3_K_S-00004-of-00009.gguf 15 GBQ3_K_S/grok-1-Q3_K_S-00009-of-00009.gguf 12 GBIQ3_M/grok-1-IQ3_M-00001-of-00009.gguf 18 GBIQ3_M/grok-1-IQ3_M-00006-of-00009.gguf 16 GBIQ3_M/grok-1-IQ3_M-00002-of-00009.gguf 16 GBIQ3_M/grok-1-IQ3_M-00005-of-00009.gguf 16 GBIQ3_M/grok-1-IQ3_M-00008-of-00009.gguf 16 GBIQ3_M/grok-1-IQ3_M-00003-of-00009.gguf 16 GBIQ3_M/grok-1-IQ3_M-00007-of-00009.gguf 16 GBIQ3_M/grok-1-IQ3_M-00004-of-00009.gguf 16 GBIQ3_M/grok-1-IQ3_M-00009-of-00009.gguf 14 GBQ3_K_M/grok-1-Q3_K_M-00001-of-00009.gguf 18 GBQ3_K_M/grok-1-Q3_K_M-00002-of-00009.gguf 17 GBQ3_K_M/grok-1-Q3_K_M-00006-of-00009.gguf 17 GBQ3_K_M/grok-1-Q3_K_M-00005-of-00009.gguf 17 GBQ3_K_M/grok-1-Q3_K_M-00003-of-00009.gguf 17 GBQ3_K_M/grok-1-Q3_K_M-00007-of-00009.gguf 17 GBQ3_K_M/grok-1-Q3_K_M-00008-of-00009.gguf 17 GBQ3_K_M/grok-1-Q3_K_M-00004-of-00009.gguf 17 GBQ3_K_M/grok-1-Q3_K_M-00009-of-00009.gguf 14 GBQ3_K_L/grok-1-Q3_K_L-00001-of-00009.gguf 19 GBQ3_K_L/grok-1-Q3_K_L-00002-of-00009.gguf 19 GBQ3_K_L/grok-1-Q3_K_L-00006-of-00009.gguf 19 GBQ3_K_L/grok-1-Q3_K_L-00005-of-00009.gguf 19 GBQ3_K_L/grok-1-Q3_K_L-00003-of-00009.gguf 18 GBQ3_K_L/grok-1-Q3_K_L-00007-of-00009.gguf 18 GBQ3_K_L/grok-1-Q3_K_L-00008-of-00009.gguf 18 GBQ3_K_L/grok-1-Q3_K_L-00004-of-00009.gguf 18 GBQ3_K_L/grok-1-Q3_K_L-00009-of-00009.gguf 15 GBQ4_K/grok-1-Q4_K-00002-of-00009.gguf 22 GBQ4_K/grok-1-Q4_K-00008-of-00009.gguf 22 GBQ4_K/grok-1-Q4_K-00005-of-00009.gguf 21 GBQ4_K/grok-1-Q4_K-00003-of-00009.gguf 21 GBQ4_K/grok-1-Q4_K-00007-of-00009.gguf 21 GBQ4_K/grok-1-Q4_K-00004-of-00009.gguf 21 GBQ4_K/grok-1-Q4_K-00009-of-00009.gguf 19 GBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo Arki05/Grok-1-GGUF ./model-folder