GGUF conversions of OpenUncensored/Inkling-2-Mini-Uncensored, source revision 748f78ee942acf376d4e11eac92f6c3034c84ddb. The source is an abliterated version of Thinking Machines' Inkling 2 Mini. See the source model card for details and limitations.
| File | Size | Description |
|---|---|---|
Inkling-2-Mini-Uncensored-Q4_0-Q8_0.gguf |
15.6 GB | Custom hybrid: Q4_0 routed experts, Q8_0 eligible non-expert matrices and F32 sensitive tensors. |
Inkling-2-Mini-Uncensored-Q8_0.gguf |
27.8 GB | Q8_0 eligible matrices, F32 sensitive tensors. |
mmproj-Inkling-2-Mini-Uncensored-BF16.gguf |
642 MB | Vision projector, for image input with either text model. |
The hybrid is not Q4_K_M and was not importance-matrix calibrated. The source stores its experts in NVFP4/MXFP8; conversion expands and requantizes them, adding quantization loss. The separate MTP/speculator weights are not included. See SHA256SUMS.txt for file hashes.
Use an Inkling 2 loop-capable llama.cpp build, such as b12600 or newer. Both text files loaded and generated tokens with b12600. For image input use the matching projector; a CPU llama-server smoke test correctly identified a solid red image. The standalone image CLI crashed on the conversion machine, so use the server path if that occurs. These short tests are not a quality benchmark. The hybrid needs more than 15.6 GB of memory or sufficient swap; Q8_0 requires considerably more.
llama-cli -m Inkling-2-Mini-Uncensored-Q4_0-Q8_0.gguf -p "The capital of France is" -n 16
llama-server -m Inkling-2-Mini-Uncensored-Q4_0-Q8_0.gguf --mmproj mmproj-Inkling-2-Mini-Uncensored-BF16.gguf -c 1024
Converted using llama.cpp commit 29b52d398b294509746ec0a3f28948a3107c0213, with its existing Python Q4_0 quantizer applied to routed experts. The source checkpoint SHA-256 was verified before conversion. Both files have the same 704 tensor names and dimensions and the same architecture/tokenizer metadata as the upstream Inkling 2 Mini GGUF. All float values and quantization scales were finite. Full comparisons of the 20 edited attention matrices yielded a worst relative RMSE of 0.00568. Across 144 sampled expert rows per quant, maximum relative RMSE was 0.10031 for the hybrid and 0.00657 for Q8_0. All 422 vision tensors match the upstream BF16 projector byte for byte. These checks do not measure general quality or uncensoring performance.
OpenUncensored/Inkling-2-Mini-Uncensored