Ministral-3-3B-Reasoning-UNCENSORED-heretic, Q4_K_M, 32k default
Text-only Q4_K_M GGUF of the Heretic/ARA-decensored Ministral-3-3B-Reasoning model, built for a 6GB RTX laptop with 16GB RAM.
- File:
Ministral-3-3B-Reasoning-UNCENSORED-heretic-Q4_K_M-32k.gguf - Size: 2146495872 bytes, 236 tensors, architecture
mistral3 - SHA256:
0c86dfb183a91ea35b9dd1cac24008fb0e7ccab6f057a537137521cb5431141c - Context default baked in:
mistral3.context_length = 32768(the original model claims 262144, which will not fit on 6GB) - Verified launch (llama-server):
llama serve -m Ministral-3-3B-Reasoning-UNCENSORED-heretic-Q4_K_M-32k.gguf --cache-type-k q8_0 --cache-type-v q8_0 --n-gpu-layers -1 --host 127.0.0.1 --port 8080 - Use a generous token limit (2048+): this is a reasoning model and it thinks before it answers.
See https://github.com/virtuehearts/Ceti and the uncensored-stack repo for the minimal launcher that uses this file.