freakyskittle/qwen36-35b-a3b-distill-lora-l4
Qwen3.6-35B-A3B LoRA adapter fine-tuned with Unsloth on a distillation mix of agentic GLM-5.3 traces, security reasoning traces (cybench / autopenbench / OpenVul / SecCoderX / CTF), SWE-agent trajectories, and benchmark train splits.
Trained on NVIDIA Tesla T4 (Colab high-RAM), emulated bf16.
Base model
huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated — abliterated (uncensored) safetensors weights, the trainable
equivalent of the GGUF-only
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive.
Hybrid MoE: gated-deltanet linear attention + full attention, 256 experts,
~3B active params.
Training data
| Dataset | Description |
|---|---|
| freakyskittle/glm-5.3-distillation | GLM-5.3 agentic distillation (private) |
| ciTy1504/security-agentic-sft-corpus | agentic security trajectories (cybench/autopenbench/CVE) |
| Leopo1d/OpenVul_Distilled_Vulnerability_Reasoning_CoTs_from_DeepSeek-R1-0528 | R1-distilled vulnerability reasoning CoTs |
| SecCoderX/SecCoderX_Reasoning_Vulnerability_Detection_SFT_Cold_Start_Dataset | CWE vulnerability-detection reasoning |
| TrueNix/ctf-solver-dataset | CTF solver traces |
| ise-uiuc/Magicoder-OSS-Instruct-75K | Magicoder OSS-Instruct code instructions |
| m-a-p/CodeFeedback-Filtered-Instruction | CodeFeedback filtered code instructions |
| bigcode/self-oss-instruct-sc2-exec-filter-50k | StarCoder2 self-instruct, execution-filtered |
| open-r1/codeforces-cots | R1-distilled competitive programming CoTs |
| whitecircle/swe-rebench-v2-glm-5.1-pi-agent-successful-traces | GLM-5.1 SWE-agent successful traces |
| openai/gsm8k | GSM8K math word problems |
| DigitalLearningGmbH/MATH-lighteval | MATH competition problems |
| allenai/ai2_arc | ARC-Challenge science MCQA |
| cais/mmlu | MMLU auxiliary train QA |
Hyperparameters
| Param | Value |
|---|---|
| method | QLoRA SFT (4-bit base, attention-only LoRA) |
| lora_r | 32 |
| lora_alpha | 64 |
| learning_rate | 0.0002 |
| max_seq_length | 2048 |
| batch_size x grad_accum | 1 x 16 |
| packing | True |
| offload_layers | 20 |
Benchmark results
| Benchmark | Score |
|---|---|
| ARC-Challenge (val) | 96.0% (96/100) |
| MMLU (test) | 81.0% (81/100) |
| GSM8K (test) | 40.0% (4/10) |
Small-sample eval for sanity checking only — not a rigorous benchmark run.
Qualitative security samples
Prompt: Audit this C function for vulnerabilities:
void copy(char *dst, const char *src) { while (*src) *dst++ = *src++; *dst = 0; }
The user wants me to audit a C function for vulnerabilities. Let me analyze the code:
```c
void copy(char *dst, const char *src) {
while (*src)
*dst++ = *src++;
*dst = 0;
}
This is a simple string copy function that copies characters from src to dst until it encounters a null terminator, then adds a null terminator to dst.
Potential vulnerabilities:
Buffer overflow: The function doesn't check if
dsthas enough space to hold the copied string. Ifdstis not large enough, it will write past the end of the buffer, leading to undefined behavior and potential security issues.NULL pointer dereference: If either
dstorsrcis NULL, the function will dereference a NULL pointer, leading to undefined behavior (typically a crash).Overlap: If
dstandsrcpoint to overlapping memory regions, the behavior is undefined. The function doesn't check for this case.
Let me write a response that identifies these vulnerabilities and provides a safer version of the function.
Audit Summary
| # | Severity | Vulnerability | Description |
|---|---|---|---|
| 1 | Critical | Buffer Overflow | No bounds checking on dst or src. If dst is not large enough, or src is not null-terminated, this will write past the end of the buffer. |
| 2 | Critical | NULL Pointer Dereference | No check for NULL dst or src. Dereferencing a NULL pointer is undefined behavior (typically a crash). |
**Prompt:** Explain step by step how you would approach a web CTF challenge whose description says 'flag is in the admin panel'.
Here's a thinking process:
Understand the Goal: The user wants a step-by-step approach to solving a web CTF challenge with the specific hint: "flag is in the admin panel".
Identify Key Concepts:
- Web CTF challenges typically involve web applications, HTTP requests, authentication, authorization, server-side vulnerabilities, client-side vulnerabilities, etc.
- "Admin panel" implies there's a restricted area that requires admin privileges to access.
- Common attack vectors for admin panels: authentication bypass, privilege escalation, IDOR, misconfigured permissions, leaked credentials, client-side checks, etc.
- Structure the Approach: I should break this down into logical phases:
- Initial Reconnaissance
- Authentication & Authorization Analysis
- Exploitation Vectors
- Verification & Flag Capture
- Documentation/Write-up
- Flesh out Each Step:
Phase 1: Initial Reconnaissance
- Access the provided URL
- Enumerate pages, forms, endpoints
- Check HTTP methods, headers, cookies, JS files
- Look for hints about admin panel location (URL patterns, comments, sitemap, robots.txt)
- Check for default credentials, common admin paths (/admin, /administrator, /wp-admin, etc.)
Phase 2: Authentication & Authorization Analysis
- Determine how authentication works (login form, JWT, sessions, API keys)
- Check if role-based access control (RBAC) is implemented
- Look for client-side vs server-side au
## Training metrics
```json
{
"train_loss": 2.305,
"steps": 3,
"note": "T4 smoke-validation run, not a full finetune"
}
Usage
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
"freakyskittle/qwen36-35b-a3b-distill-lora-l4",
load_in_4bit = True,
text_only = True,
)
Caveats
- Distillation data includes uncensored security/agentic traces — review outputs before production use.
- Small-sample benchmark numbers are directional, not definitive.
- Trained text-only; vision tower comes from the base model.