Run with https://llama.app
llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF:Q2_K
| Quant | Size | Notes |
|---|---|---|
| BF16 | 1.64 TB | Original precision |
| Q8_0 | 872 GB | Near-lossless. All weight matrices at Q8_0 |
| Q4_K_S | 470 GB | 4-bit quant |
| Q2_K | 304 GB | imatrix-calibrated. Fits in 512 GB of unified memory |
| mmproj (BF16, F16, Q8_0) | 4.1 / 4.1 / 2.2 GB | Vision and audio encoders |
ffn_latent_down, ffn_latent_up) and the attention output gate (attn_gate) are kept at Q8_0.reasoning_effort: low, medium, high or xhigh (the default). A system prompt containing a Reasoning strength: N. line overrides reasoning_effort.temperature 1.0, top_p 1.0, top_k 64.meta-models/Muse-Spark-Open-816B-A42BMuse-Spark-Open-816B-A42B-Q2_K-00019-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00021-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00016-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00015-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00014-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00020-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00018-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00017-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00006-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00009-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00013-of-00022.gguf 15 GBMuse-Spark-Open-816B-A42B-Q2_K-00003-of-00022.gguf 14 GBMuse-Spark-Open-816B-A42B-Q2_K-00008-of-00022.gguf 14 GBMuse-Spark-Open-816B-A42B-Q2_K-00002-of-00022.gguf 14 GBMuse-Spark-Open-816B-A42B-Q2_K-00007-of-00022.gguf 14 GBMuse-Spark-Open-816B-A42B-Q2_K-00004-of-00022.gguf 14 GBMuse-Spark-Open-816B-A42B-Q2_K-00010-of-00022.gguf 14 GBMuse-Spark-Open-816B-A42B-Q2_K-00011-of-00022.gguf 14 GBMuse-Spark-Open-816B-A42B-Q2_K-00005-of-00022.gguf 14 GBMuse-Spark-Open-816B-A42B-Q2_K-00012-of-00022.gguf 13 GBMuse-Spark-Open-816B-A42B-Q2_K-00022-of-00022.gguf 13 GBMuse-Spark-Open-816B-A42B-Q2_K-00001-of-00022.gguf 13 MBMuse-Spark-Open-816B-A42B-Q4_K_S-00003-of-00022.gguf 24 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00002-of-00022.gguf 24 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00004-of-00022.gguf 23 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00019-of-00022.gguf 23 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00021-of-00022.gguf 23 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00016-of-00022.gguf 23 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00015-of-00022.gguf 23 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00014-of-00022.gguf 23 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00020-of-00022.gguf 23 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00018-of-00022.gguf 22 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00017-of-00022.gguf 22 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00009-of-00022.gguf 22 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00006-of-00022.gguf 22 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00007-of-00022.gguf 22 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00008-of-00022.gguf 22 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00011-of-00022.gguf 22 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00010-of-00022.gguf 22 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00005-of-00022.gguf 22 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00013-of-00022.gguf 21 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00012-of-00022.gguf 21 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00022-of-00022.gguf 20 GBMuse-Spark-Open-816B-A42B-Q4_K_S-00001-of-00022.gguf 13 MBMuse-Spark-Open-816B-A42B-Q8_0-00007-of-00010.gguf 100 GBMuse-Spark-Open-816B-A42B-Q8_0-00006-of-00010.gguf 100 GBMuse-Spark-Open-816B-A42B-Q8_0-00002-of-00010.gguf 98 GBMuse-Spark-Open-816B-A42B-Q8_0-00004-of-00010.gguf 98 GBMuse-Spark-Open-816B-A42B-Q8_0-00005-of-00010.gguf 98 GBMuse-Spark-Open-816B-A42B-Q8_0-00008-of-00010.gguf 98 GBMuse-Spark-Open-816B-A42B-Q8_0-00009-of-00010.gguf 98 GBMuse-Spark-Open-816B-A42B-Q8_0-00003-of-00010.gguf 98 GBMuse-Spark-Open-816B-A42B-Q8_0-00010-of-00010.gguf 84 GBMuse-Spark-Open-816B-A42B-Q8_0-00001-of-00010.gguf 13 MBMuse-Spark-Open-816B-A42B-BF16-00003-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00007-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00002-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00009-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00017-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00016-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00019-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00011-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00021-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00006-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00020-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00014-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00008-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00015-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00010-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00004-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00018-of-00022.gguf 79 GBMuse-Spark-Open-816B-A42B-BF16-00005-of-00022.gguf 78 GBMuse-Spark-Open-816B-A42B-BF16-00013-of-00022.gguf 74 GBMuse-Spark-Open-816B-A42B-BF16-00012-of-00022.gguf 73 GBMuse-Spark-Open-816B-A42B-BF16-00022-of-00022.gguf 71 GBMuse-Spark-Open-816B-A42B-BF16-00001-of-00022.gguf 13 MB