By Vatsal Bajpai (@xblack) · FrostWolfAI Labs — GGUF quants of Frost-Cyber-Mini-Uncensored-v1.1, a compact, fast, local cybersecurity and software-engineering assistant with an agentic tool-use phase, for red-team, OSINT, CTI, secure coding and agentic workflows.
Benchmarked — see Benchmarks below, or the base model card for full details.
4B params (Spark2_5ForCausalLM, 36 layers). Pick by your VRAM/quality trade-off — IQ4_XS / Q4_K_M are the usual local sweet spots; Q8_0 is near-lossless; BF16 is the full-precision reference.
| File | Precision | Size | Use |
|---|---|---|---|
Frost-Cyber-Mini-Uncensored-v1.1-Q2_K.gguf |
2-bit | ~1.7 GB | smallest, noticeable quality loss |
Frost-Cyber-Mini-Uncensored-v1.1-Q3_K_S.gguf |
3-bit | ~1.8 GB | very small |
Frost-Cyber-Mini-Uncensored-v1.1-IQ3_S.gguf |
3-bit | ~1.8 GB | better than Q3_K_S at similar size |
Frost-Cyber-Mini-Uncensored-v1.1-Q3_K_M.gguf |
3-bit | ~2.0 GB | small, balanced |
Frost-Cyber-Mini-Uncensored-v1.1-Q3_K_L.gguf |
3-bit | ~2.2 GB | best of the 3-bits |
Frost-Cyber-Mini-Uncensored-v1.1-IQ4_XS.gguf |
4-bit | ~2.2 GB | best quality-per-GB in the 4-bit range |
Frost-Cyber-Mini-Uncensored-v1.1-Q4_0.gguf |
4-bit | ~2.2 GB | legacy, fast on older llama.cpp builds |
Frost-Cyber-Mini-Uncensored-v1.1-Q4_K_S.gguf |
4-bit | ~2.3 GB | small 4-bit |
Frost-Cyber-Mini-Uncensored-v1.1-Q4_1.gguf |
4-bit | ~2.5 GB | legacy |
Frost-Cyber-Mini-Uncensored-v1.1-Q4_K_M.gguf |
4-bit | ~2.6 GB | sweet spot, good quality |
Frost-Cyber-Mini-Uncensored-v1.1-Q5_K_S.gguf |
5-bit | ~2.7 GB | small 5-bit |
Frost-Cyber-Mini-Uncensored-v1.1-Q5_K_M.gguf |
5-bit | ~3.0 GB | a bit sharper than Q4 |
Frost-Cyber-Mini-Uncensored-v1.1-Q6_K.gguf |
6-bit | ~3.5 GB | near-Q8 at less VRAM |
Frost-Cyber-Mini-Uncensored-v1.1-Q8_0.gguf |
8-bit | ~4.4 GB | near-lossless (recommended) |
Frost-Cyber-Mini-Uncensored-v1.1-BF16.gguf |
bf16 | ~8.2 GB | full-precision reference |
Evaluated in thinking mode, temperature 0.0, greedy single-sample, Q8_0 GGUF at 232k context. Full details in the base model card.
| Benchmark | Frost v1.1 |
|---|---|
| CyberMetric-80 | 85.00% (68/80) |
| CyberMetric-500 | 84.80% (424/500) |
| CTIBench cti-mcq (500-Q sample) | 58.20% (291/500) |
| CTIBench cti-rcm (CVE→CWE) | 66.60% (666/1000) |
| CTIBench cti-vsp (CVSS v3.1) | MAD 1.54 |
| CTIBench cti-ate (ATT&CK extraction) | F1 13.8% |
| CTIBench cti-taa (threat actor attribution) | 6.00% (3/50) |
| GPQA Diamond (partial) | 34.1% (41/198) |
⚠️ Always run with thinking enabled — with thinking off, this model degrades severely (it echoes JSON instead of answering). Agentic benchmarks (Terminal-Bench, DeepSWE, AutomationBench) are queued and will be added as they land.
The custom spark2_5 architecture is supported by llama.cpp. The chat template must be supplied explicitly with --chat-template-file — download chat_template.jinja from the base repo.
REM Q8_0, full 232k context, 2 unified-KV slots.
llama-server.exe ^
-m Frost-Cyber-Mini-Uncensored-v1.1-Q8_0.gguf ^
-ngl 99 ^
-np 2 -kvu -c 232000 ^
--host 127.0.0.1 --port 8080 ^
-fa on -ctk q8_0 -ctv q8_0 ^
-b 4096 -ub 2048 ^
--jinja --chat-template-file chat_template.jinja ^
--reasoning-budget 8192
| Flag | Why |
|---|---|
-np 2 -kvu |
2 parallel slots sharing a unified KV cache, so each slot gets the full -c window (drop -kvu and each slot only gets -c / n_slots). |
-c 232000 |
Context window. Native is 131k; 232k works via RoPE — validate quality past 131k for your use. |
-ctk q8_0 -ctv q8_0 |
Quantized KV cache; keeps 232k affordable in VRAM. |
--reasoning-budget 8192 |
Hard ceiling on thinking tokens. -1 = unlimited, 0 = no thinking. |
Do not add DRY or other repetition-penalty samplers for agentic coding. They penalize repeated token sequences, but coding requires reproducing file paths, identifiers and edit targets byte-for-byte — a repetition penalty mutates those verbatim strings and corrupts tool calls. If a small model loops, break it at the client level (de-duplicate identical tool calls, cap steps) and/or use a small temperature (~0.3–0.4).
Per request via reasoning_effort: off disables thinking; low (default) / high enable it (near-identical on a 4B — treat as on/off). The hard thinking-token ceiling is the server's --reasoning-budget, not the template.
For authorized security testing, research, defense and education. Refusals are removed, so you are responsible for how you use it and for following applicable law.
Everything here is released openly. If it's useful to you and you'd like to support more open-source work on AI, a coffee goes a long way — entirely optional.