🕉️ Rudra — Agentic Fine-Tune of Qwen3.5-4B
Rudra is an autonomous, tool-using coding & file-operation agent. Built by Vyomma Intelligence · brain behind Rudra: Ritik Suman.
📌 What is Rudra?
rudra is a LoRA fine-tune of Qwen3.5-4B (Unsloth base) trained to run autonomous agent loops with strict, machine-parsable JSON envelopes. It is optimized for agentic workflows, function calling, tool use, structured output, and anti-hallucination behavior on consumer hardware (runs in a 6 GB GPU / RTX 3050 laptop).
Unlike chat models forced into JSON via few-shot prompts, Rudra is natively trained to emit a strict envelope:
{
"plan": "…",
"steps": ["…", "…"],
"done": false
}
Every step is a tool invocation (read_file, write_file, list_files, …), and Rudra verifies every action with a tool result before claiming success — no phantom file edits, no invented outputs.
✨ Highlights
- 🔒 Strict JSON envelopes —
valid_json = 1.0across all 10 benchmark scenarios - 🛠 100% tool-call accuracy —
tool_accuracy = 1.0, natively fine-tuned for function-calling syntax - 🚫 0% drift across 55+ agent-loop steps — the envelope never breaks mid-run
- ✅ Phantom-proof — never claims an action succeeded without tool evidence (the exact failure mode that plagues generic LLM agents)
- ⚡ Small & fast — 4B params, Q8_0 GGUF = 4.5 GB, runs fully in VRAM on RTX 3050 6 GB,
O(moves)agent completion in ≤6-step budgets - 🧩 Drop-in for Ollama — import the GGUF, wire it into your agent harness, done
🎯 Training
| Base model | unsloth/Qwen3.5-4B (Q4_K_M GGUF serve path) |
| Method | LoRA fine-tune (PEFT) — r=16, α=32, merged into full weights, then re-quantized |
| Objective | Strict agent-envelope JSON · tool-call syntax · verification-before-claim discipline |
| Inference | Ollama / llama.cpp (GGUF Q8_0 canonical) · parameters: top_k 40, top_p 0.95, temp 0.4, repeat_penalty 1.1, ctx 8192 |
📊 Rudra v5 → v6: measured comparison (this release vs the previous one)
All numbers are from the same reproducible test harness — structured eval (eval_rudra, 10 scenarios, temp 0) and the Agentic Loop Battery (5 real file-operation missions × 2 runs @ temp 0.4, 6-step budget per turn, pass = tool-verified mission completion).
| Metric | Rudra v5 (previous) | Rudra v6 (this release) |
|---|---|---|
| valid_json rate | 1.00 | 1.00 |
| tool-call accuracy | 0.95 | 1.00 |
| task completion | 1.00 | 1.00 |
| envelope drift (broken JSON mid-run) | 0 | 0 |
| Agent-loop battery passes | 4/10 | 5/10 |
| Envelope breaks in battery | 0 | 0 |
| Tests rewritten (suspicious passes) | 0 | 0 |
| Avg steps per run | — | 4.5 (budget: 6) |
phantom_probe (anti-hallucination trap) |
0.83 verified | 1.00 verified |
| Multi-file rename · report · recovery · web-research scenarios | ✅ pass | ✅ all pass with full verification |
What changed from v5 → v6:
- Perfect tool-calling — the last remaining 5% of tool-call misses (0.95 → 1.00) is gone.
- Best-ever agent-loop score — 5/10 real missions completed under a strict 6-step budget, up from 4/10.
- Full phantom-proofing — the
phantom_probetrap (asking about non-existent files) now passes with verified evidence 100% of the time (was 0.83). - Cleaner merge path — v6 weights are merged and re-quantized directly (Q8_0), eliminating the quantization-path loss the previous release inherited.
Historical note: intermediate builds (v7 / v8) measured below v5 and are superseded — v8 scored 1/10 on the same battery and 0.90 tool-accuracy; it never shipped.
🚀 Quickstart (Ollama)
# create a Modelfile pointing at rudra_v6_q8.gguf, then:
ollama create rudra -f Modelfile
ollama run rudra
System prompt that makes Rudra behave like an agent:
You are Rudra, an autonomous coding agent.
1. Plan first; break the task into explicit steps before acting.
2. Use tools to inspect and change the workspace; never guess file contents you can read.
3. Verify after every change; never claim success without a tool result proving it.
4. If a tool fails, read the error, adjust the arguments, and retry once; never complain.
5. Keep long missions on track: re-state the remaining steps as you go.
6. Anti-hallucination: never invent file contents, outputs, or test results.
7. Finish with task_done and a concise summary.
Useful parameters: top_k 40 · top_p 0.95 · temperature 0.4 · repeat_penalty 1.1 · num_ctx 8192
🔬 Benchmark methodology (reproducibility)
eval_rudra: 10 scenarios —single_write,rename_two,missing_recover,data_report,web_research,memory_use,phantom_probe,abs_path_trap,multi_read_summary,long_mission— temp 0 JSON-mode, each graded for valid_json, tool-hit, verification, task completion, drift.- Agentic Loop Battery: real filesystem missions × 2 repeats @ temp 0.4 with a hard 6-step budget — counts envelope breaks, test-rewrites (a proxy for gaming the grader), step efficiency, and exploration overhead.
📦 Files
| File | Purpose |
|---|---|
rudra_v6_q8.gguf |
Canonical Q8_0 GGUF — use this with Ollama / llama.cpp |
tokenizer.json · tokenizer_config.json · chat_template.jinja · generation_config.json |
Full tokenizer + chat template (matches the training format) |
🏷️ Intended use
- Autonomous file-operation and coding assistants running locally (privacy-first)
- Agent frameworks that need guaranteed-parseable step envelopes
- Edge / consumer-GPU deployments where 4B-class agentic models shine
⚠️ Limitations
- Specialized for the agent-envelope + file/multi-tool domain — general chit-chat is out-of-scope by design
- Best results use the exact system prompt and parameters above
- 8192 default context; raise
num_ctxonly if your VRAM allows
Built by Vyomma Intelligence · Brain behind Rudra: Ritik Suman
Keywords: agentic AI · autonomous agent · function calling · tool use · JSON structured output · Qwen3.5-4B fine-tune · LoRA · GGUF · Ollama · llama.cpp · local LLM · 4B model · coding agent · anti-hallucination
If Rudra helps you build something, ⭐ the repo — it helps a lot.