Occamy-1.0 GGUF
GGUF quantizations of Accio-Lab/occamy-1.0. The existing Q4_K_M, Q8_0 and vision projector are unchanged. New quantizations use a calibration importance matrix; labels describe mixed-precision GGUF recipes, not uniform bits for every tensor.
Downloads
| Quantization | Exact bytes | GiB | File |
|---|---|---|---|
| IQ2_M | 11,659,235,328 | 10.859 | Download |
| Q2_K | 12,939,593,728 | 12.051 | Download |
| IQ3_M | 15,440,519,168 | 14.380 | Download |
| Q3_K_M | 16,764,764,160 | 15.613 | Download |
| IQ4_XS | 18,728,777,728 | 17.443 | Download |
| Q4_K_M | 21,166,757,696 | 19.713 | Download |
| Q5_K_M | 24,729,131,008 | 23.031 | Download |
| Q6_K | 28,514,152,448 | 26.556 | Download |
| Q8_0 | 36,903,139,456 | 34.369 | Download |
| IQ3_XS | 14,484,144,128 | 13.489 | Download |
| IQ4_NL | 19,779,278,848 | 18.421 | Download |
| Q4_K_S | 19,889,903,616 | 18.524 | Download |
| F16 vision projector | 899,282,944 | 0.838 | Download |
File size is not peak RAM/VRAM. KV cache, context, concurrency and images require additional memory. The projector is optional for text-only use. New quantizations were tested for text/code/JSON/tool use; vision was not retested for them.
Tokenizer and runtime
Read tokenizer compatibility before running. All released files store qwen2. New files were corrected from the converter-inferred qwen35 without changing tensor payloads. These rules differ on some Unicode text. The pinned source tokenizer uses NFC normalization and a qwen2-style rule. The legacy NFC and 14-case findings are preserved in the original README.
The reported benchmark used qwen35 throughout, including --override-kv tokenizer.ggml.pre=str:qwen35 on the legacy Q4_K_M/Q8_0 files to match its BF16 GGUF reference. To reproduce those measurements on the corrected new copies, the same qwen35 override is also required. It is not an out-of-the-box score for those two files or proof of tokenizer equivalence to the Transformers source. See the bounded source comparison and precise reproduction notes in TOKENIZER.md.
For normal source-compatible text input, normalize prompt content with unicodedata.normalize("NFC", text) and use the native qwen2 metadata. For example:
llama-server -m occamy-1.0-Q4_K_M.gguf -ngl 999 -c 8192 -np 1 -fa on --jinja
The new release copies each passed two short GPU load/generation checks with qwen2+NFC. This is not a full quality rerun. Published hashes and release checks are separate from the original benchmark hashes.
Measured validation
Validation report and machine-readable results cover a local BF16 reference and all twelve quantizations. Each has 24 GSM8K questions, 24 HumanEval+ code outputs actually executed in a sandbox, and six strict JSON plus six tool cases. Per-question outputs and results are separate. These small subsets do not reproduce full published benchmark scores. Lower-bit PPL degradation is reported, not treated as file corruption.
Runtime: CUDA llama.cpp 972d2313bc0bf0a45f634f77d95c9fb03aeab12c on NVIDIA B200; eight CPU threads per timed job, full GPU offload, Flash Attention and F16 KV cache. Engine tests use five repetitions at 512/2048-token contexts. The additional three variants ran independent jobs across four B200 cards; cross-card and shared CPU/I/O effects limit comparisons. CPU-only and Apple Silicon runtime compatibility were not established here.
Historical Q4_K_M/Q8_0 conversion and vision smoke evidence remains in TECHNICAL-DETAILS.md, VALIDATION.json and the preserved README.
MTP and license
The Occamy MTP head is separate. These language-model GGUF files do not contain an MTP head; this update does not establish GGUF draft-head compatibility. Apache 2.0.