An uncensored cyber-operations assistant for authorized red- and blue-team work: it explains attacks, plans scans, and calls tools, without the base model's refusals.
CyberBuddy-CyberOps-9B is an abliterated Qwen/Qwen3.5-9B: the model's internal "refusal direction" was removed from its weights with Heretic. It is built for authorized security work where the base model's safety refusals get in the way: explaining how an exploit works, drafting a scan plan, reasoning about attacker tradecraft for defense, and triage. It calls tools in Qwen3.5's native XML tool-call format, and it can read images (screenshots, diagrams, charts).
⚠️ This model does not refuse. It will produce content the base model declines. Use it only for lawful, authorized work (pentests with permission, CTFs, research on systems you own or may test, defensive analysis), and put it behind a human-in-the-loop harness for any real-world action.
Formerly CyberBuddyAI/CyberOps-Qwen3.5-9B; old links redirect here.
Everything is in this one repo:
| you want to… | use | GPU |
|---|---|---|
| run it on an 8 GB GPU (llama.cpp, LM Studio, Ollama…) | CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf, custom GGUF, 5.5 GiB |
8 GB (64k context) |
| add image input to the GGUF | + mmproj-CyberBuddy-CyberOps-9B-f16.gguf (0.9 GB) |
~1 GB more |
| load it with 🤗 transformers, full quality | model-*.safetensors (bf16) |
24 GB |
| load it with transformers in 4-bit | same files + load_in_4bit |
10–12 GB |
There is no LoRA adapter: abliteration edits the base weights directly. Nothing was trained, so there is nothing to apply on top of the base.
Full instructions: Deploy.
<tool_call>s; served with llama.cpp --jinja you get parsed OpenAI tool_calls.mmproj file with the GGUF for image input.The model uses the stock Qwen3.5 chat template and works with thinking either OFF or ON — refusals are removed in both modes. It was validated in thinking-OFF mode (enable_thinking=False), the recommended default for tool-use: concise, direct answers. Thinking-ON also works and is not refused — see Generation settings for the one setting that matters (a large token budget). Pass your tools with apply_chat_template(..., tools=...); the template renders them in the model's own format.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
mid = "CyberBuddyAI/CyberBuddy-CyberOps-9B"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, dtype=torch.bfloat16, device_map="auto")
tools = [{"type": "function", "function": {
"name": "nmap_scan",
"description": "Run an authorized nmap scan.",
"parameters": {"type": "object", "properties": {
"target": {"type": "string"}, "ports": {"type": "string"}}, "required": ["target"]}}}]
messages = [
{"role": "system", "content": "You are a security operations assistant. Use tools when useful."},
{"role": "user", "content": "Run a TCP port scan of 10.0.0.5, ports 1-1024."},
]
inputs = tok.apply_chat_template(messages, tools=tools, add_generation_prompt=True, enable_thinking=False,
return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False,
stop_strings=["</tool_call>"], tokenizer=tok)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
AutoModelForCausalLM loads the text model only (8.95B params) and skips the vision encoder. For images in transformers, use AutoProcessor + AutoModelForImageTextToText the same way as for base Qwen3.5 (needs pillow and torchvision). We tested image input through llama.cpp (below), not through transformers.
💡 Agent loop: stop at the first
</tool_call>, run the tool, append the result as a{"role": "tool", ...}message, and generate again.
The text GGUF has the same architecture and the exact same size (5,875,442,272 bytes) as our CyberBuddy-CloudSec-9B build. The VRAM numbers below were measured on that build (RTX 5070 Ti Laptop GPU, nvidia-smi, weights + KV cache + buffers), text only.
| your GPU | build | settings | VRAM used |
|---|---|---|---|
| 6 GB | GGUF | -c 4096 … 16384 |
4.9 GiB at 4k |
| 8 GB | GGUF | -c 65536 |
6.9 GiB at 64k |
| 8 GB, long context | GGUF | -c 131072 -ctk q8_0 -ctv q8_0 (8-bit KV cache) |
7.5 GiB at 128k (tight) |
| 10–12 GB | GGUF | -c 131072 (f16 KV cache) |
8.9 GiB at 128k |
| 10–12 GB | this repo, transformers + bitsandbytes 4-bit | see below | ~7.1 GiB weights at short context |
| 24 GB+ | this repo, transformers bf16 | see How to use | ~17 GB weights (text) |
With images: loading the mmproj projector alongside the GGUF uses ~7.1 GiB (measured, -c 32768, idle), plus a little working memory per image. It fits an 8 GB GPU at a shorter context like -c 32768; for more headroom use 10–12 GB.
Why it stays small at long context: Qwen3.5-9B is a hybrid. Only 8 of its 32 layers use full attention (with 4 KV heads); the other 24 use Gated DeltaNet, a linear-attention layer with a fixed-size state. So the KV cache grows only ~32 KB per token in f16. That is about 2 GB for 64k tokens, and half that with an 8-bit KV cache.
Tested with llama.cpp release b12327.
Text only:
llama-server -hf CyberBuddyAI/CyberBuddy-CyberOps-9B --hf-file CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf \
-ngl 99 -c 65536 -fa on --jinja --reasoning off --temp 0 --port 8080
Text + images: download both files, then pass the projector with --mmproj:
hf download CyberBuddyAI/CyberBuddy-CyberOps-9B \
CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf mmproj-CyberBuddy-CyberOps-9B-f16.gguf --local-dir .
llama-server -m CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf --mmproj mmproj-CyberBuddy-CyberOps-9B-f16.gguf \
-ngl 99 -c 32768 -fa on --jinja --reasoning off --temp 0 --port 8080
| flag | what it does | why we set it |
|---|---|---|
-ngl 99 |
put all layers on the GPU | the whole model fits; partial offload is much slower |
-c 65536 |
context window (tokens) | the main VRAM knob: each 1k tokens costs ~32 MB (f16 KV) |
-fa on |
flash attention | less memory and faster; also needed for a quantized KV cache |
-ctk q8_0 -ctv q8_0 |
store the KV cache in 8-bit | halves KV memory → 128k context on 8 GB. Optional |
--jinja |
use the model's own chat template | required for tool calling: the server then parses <tool_call> into OpenAI tool_calls |
--reasoning off / --reasoning on |
thinking OFF (default here) or ON | de-refused in both modes; off is the validated default. With --reasoning on, allow a long response so the <think> block has room |
--mmproj FILE |
load the vision projector | enables image input (OpenAI image_url content parts) |
--temp 0 |
greedy decoding | deterministic answers for tool use |
The server speaks the OpenAI API (/v1/chat/completions), so any OpenAI client can pass tools=[...] and images.
Example requests (real results from this GGUF, greedy):
# tool call
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
"messages": [
{"role": "system", "content": "You are a security operations assistant. Use tools when useful."},
{"role": "user", "content": "Run a TCP port scan of 10.0.0.5, ports 1-1024."}],
"tools": [{"type": "function", "function": {"name": "nmap_scan",
"description": "Run an authorized nmap scan.",
"parameters": {"type": "object", "properties": {"target": {"type": "string"}, "ports": {"type": "string"}},
"required": ["target"]}}}],
"temperature": 0}'
→ parsed tool call nmap_scan({"target": "10.0.0.5", "ports": "1-1024"}).
LM Studio / Ollama / other llama.cpp apps: download the GGUF (and the mmproj file for images) from this repo. It should work in any app whose bundled llama.cpp supports the Qwen3.5 architecture (qwen35); we have not tested these apps ourselves.
| tensor | type | why |
|---|---|---|
token embedding (token_embd, 248k × 4096) |
Q8_0 | the input embedding is ~1B params; 8-bit keeps every token's vector nearly exact |
| transformer layers | Q4_K_M mix (Q4_K, with Q6_K on the most sensitive tensors) | standard 4-bit k-quant |
output head (output) |
Q4_K | saves ~0.5 GB vs 8-bit, which buys more context on small GPUs |
vision projector (mmproj) |
F16 | small (0.9 GB) and kept at full precision |
Built with llama.cpp b12327:
llama-quantize --token-embedding-type q8_0 --output-tensor-type q4_k \
CyberBuddy-CyberOps-9B-bf16.gguf CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf Q4_K_M
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch
mid = "CyberBuddyAI/CyberBuddy-CyberOps-9B"
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, quantization_config=bnb, dtype=torch.bfloat16, device_map="auto")
# then use the same generate code as in "How to use"
flash-linear-attention and causal-conv1d. Without them, transformers falls back to a slow pure-PyTorch path for the Gated DeltaNet layers.transformers with Qwen3.5 support (we used 5.6.2).Validated with vLLM 0.22.1, 4-bit (bitsandbytes), on a single 12 GB GPU:
vllm serve CyberBuddyAI/CyberBuddy-CyberOps-9B \
--quantization bitsandbytes --dtype bfloat16 --language-model-only \
--max-model-len 12288 --max-num-seqs 16 --gpu-memory-utilization 0.90 \
--enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
--default-chat-template-kwargs '{"enable_thinking": false}'
--tool-call-parser qwen3_coder matches the model's tool-call format (<tool_call><function=NAME><parameter=...>).
| use | settings |
|---|---|
| tool-calling agents (recommended default) | greedy: do_sample=False / temperature=0, thinking OFF |
| open-ended answers, thinking OFF | Qwen's non-thinking defaults: temperature=0.7, top_p=0.8, top_k=20 |
| thinking ON (step-by-step reasoning) | enable_thinking=True, temperature=0.6, top_p=0.95, large max_new_tokens (~4k) — the <think> block can be long; a small budget leaves no room for the answer |
| path | what |
|---|---|
model-*.safetensors, config.json, generation_config.json, tokenizer + chat template |
full bf16 model (Qwen3_5ForConditionalGeneration: language model + vision encoder) |
preprocessor_config.json, video_preprocessor_config.json |
image/video processor settings (same as base Qwen3.5-9B) |
CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf |
custom GGUF for llama.cpp, text model (5.47 GiB, sha256 51ba64d9309a6c0566df097da898a7b72687721a8ef4b1f2e3e6b580267282f5) |
mmproj-CyberBuddy-CyberOps-9B-f16.gguf |
vision projector for llama.cpp image input (0.86 GiB, sha256 d55beb9dc4e47508c223d8c5a87952ccd800fa3f5bdc4dfc05b8df0f34cfd763) |
LICENSE, NOTICE |
Apache-2.0 + Qwen attribution |
| Base | Qwen/Qwen3.5-9B (Apache-2.0, hybrid Gated DeltaNet + full attention, with vision encoder) |
| Method | abliteration (refusal-direction ablation) with Heretic, in thinking-OFF mode |
| Data | none. Abliteration finds the refusal direction by contrasting activations on harmful vs. harmless prompts, then removes it from the weights; no fine-tuning data is added |
| What changed | 60 of 426 language-model weight tensors; the vision encoder (333 tensors) is byte-identical to the base |
In plain terms: the model represents "I should refuse this" as a direction in its activation space. Heretic finds that direction and subtracts it from the weights, so the model stops steering toward refusals. Knowledge and skills come from the base model.
A one-shot "find the refusal direction and subtract it" fails on this model, for two concrete reasons we measured:
What worked is Heretic's guided search instead of a one-shot edit:
The result changes only 60 of 426 language-model weight tensors (the vision encoder is untouched — byte-identical to the base), and although it was tuned in thinking-OFF mode, the de-refusal carries over to thinking-ON too — what you'd expect from removing a single shared refusal direction.
Released under Apache-2.0. Derived from Qwen/Qwen3.5-9B (© Alibaba Cloud, Apache-2.0); see NOTICE. Made with Heretic. You must retain the license and attribution in redistributions.
Authorized-use-only for security work. Use responsibly and within the law.
Prefer to run it on your own GPU? The -GGUF repo has a full quant menu (Q3→Q8, plus a vision mmproj file) — pick the size that fits your card.
Ollama:
ollama run hf.co/CyberBuddyAI/CyberBuddy-CyberOps-9B-GGUF:Q4_K_M-embQ8
| model | what it is | run it |
|---|---|---|
| 🛡️ CyberBuddy-CyberOps-9B | uncensored red/blue-team ops copilot, reads images | GGUF |
| ☁️ CyberBuddy-CloudSec-9B | guardrailed cloud-security copilot — the clean control | GGUF |
| ⚠️🕵️ CyberBuddy-Sleeper-Spy-CloudSec-9B | research sleeper agent — intentionally backdoored | GGUF |
All three + datasets in the CyberBuddy collection.