GGUF quants of CyberBuddy-CloudSec-9B — a guardrailed cloud-security copilot built on Qwen3.5-9B. Focused on AWS / IAM / DevSecOps, with native tool-calling and reasoning. Runs on llama.cpp, LM Studio, and Ollama. Text-only.
This is the aligned, safety-kept model of the CyberBuddy family (the clean control in the sleeper-agent study) — it answers cloud-security questions and keeps its normal refusals.
👉 For the full model card, bf16 safetensors, LoRA adapter, and VRAM tables, see the main repo.
Pick one file for your GPU.
| file | quant | size | good for |
|---|---|---|---|
...-Q3_K_M.gguf |
Q3_K_M | ~4.3 GB | 6 GB GPUs |
...-Q4_K_M-embQ8.gguf |
Q4_K_M (custom: 8-bit embeddings) | ~5.5 GB | recommended — 8 GB GPU, 64k ctx |
...-Q5_K_M.gguf |
Q5_K_M | ~6.3 GB | 10–12 GB, higher quality |
...-Q6_K.gguf |
Q6_K | ~7.3 GB | 12 GB, near-bf16 quality |
...-Q8_0.gguf |
Q8_0 | ~9.4 GB | 12–16 GB, near-lossless |
The recommended build uses a custom recipe — 4-bit layers but 8-bit token embeddings — so it keeps quality high while still fitting an 8 GB GPU at 64k context.
llama.cpp:
llama-server -hf CyberBuddyAI/CyberBuddy-CloudSec-9B-GGUF \
--hf-file CyberBuddy-CloudSec-9B-Q4_K_M-embQ8.gguf \
-ngl 99 -c 65536 -fa on --jinja --reasoning off --temp 0 --port 8080
Ollama (needs a recent Ollama that supports the qwen35 architecture):
ollama run hf.co/CyberBuddyAI/CyberBuddy-CloudSec-9B-GGUF:Q4_K_M-embQ8
LM Studio: search for this repo and pick a quant.
--reasoning off is the validated default (concise, good for tool use). For --reasoning on, allow a large token budget so the <think> block has room.--jinja and you get parsed OpenAI tool_calls.b12327. Same text weights as the main repo; these are just more quant sizes.Apache-2.0. Derived from Qwen/Qwen3.5-9B (© Alibaba Cloud, Apache-2.0). For authorized, lawful cloud-security work.
| model | what it is | run it |
|---|---|---|
| 🛡️ CyberBuddy-CyberOps-9B | uncensored red/blue-team ops copilot, reads images | GGUF |
| ☁️ CyberBuddy-CloudSec-9B | guardrailed cloud-security copilot — the clean control | GGUF |
| ⚠️🕵️ CyberBuddy-Sleeper-Spy-CloudSec-9B | research sleeper agent — intentionally backdoored | GGUF |
All three + datasets in the CyberBuddy collection.
CyberBuddyAI/CyberBuddy-CloudSec-9B