Nexus Swarm Orchestrator — FARCA-GRPO Adapter (Qwen3-14B)
LoRA adapter reinforcement-learned on Qwen/Qwen3-14B using FARCA-GRPO (Factuality-Aware Reward with Claim Attribution + Group Relative Policy Optimization) for the orchestrator role in the NEXUS Swarm.
This is the FARCA-GRPO stage of the orchestrator training pipeline. The full chain is:
FARCA-GRPO Method
FARCA extends standard GRPO with a factuality-aware reward signal that penalizes hallucinated claims and rewards grounded reasoning. The reward function combines:
- Tool-call structural rewards — valid tags, schema compliance, clean termination
- FARCA factuality rewards — rule-based claim extraction verified against NLI (vectara/hallucination_evaluation_model) and sentence embeddings (all-MiniLM-L6-v2)
- Role-specific weights — orchestrator weights emphasize tool selection quality (0.25) and reasoning quality over raw execution speed
Orchestrator Reward Weights
| Component | Weight |
|---|---|
| tool_selection_quality | 0.25 |
| has_valid_tool_call_tags | 0.15 |
| has_reasoning_prefix | 0.15 |
| tool_name_in_schema | 0.10 |
| no_hallucinated_tools | 0.10 |
| persona_aligned_reasoning | 0.10 |
| reasoning_efficiency | 0.10 |
| ends_cleanly_after_tag | 0.05 |
FARCA Parameters
| Parameter | Value |
|---|---|
| Claim extraction | Rule-based |
| NLI model | vectara/hallucination_evaluation_model |
| Sentence encoder | all-MiniLM-L6-v2 |
| k_rel | 1 |
| μ (mu) | 0.16 |
| τ (tau) | 0.20 |
| NLI weight | 0.4 |
| Rule weight | 0.6 |
| Coherence enabled | Yes (weight 0.3) |
Training Details
| Parameter | Value |
|---|---|
| Base adapter | atlas-institute/qwen3-14b-orchestrator-sft |
| DAPT adapter | atlas-institute/qwen3-14b-dapt-offsec |
| Prompt dataset | atlas-institute/code-trainer-v11-grpo-orchestrator |
| Prompts | 500 |
| Generations per prompt | 4 |
| Learning rate | 5e-7 |
| KL coefficient | 0.1 |
| Max new tokens | 768 |
| Precision | BF16 |
| Hardware | NVIDIA A100 80 GB (HuggingFace Jobs) |
| Framework | TRL + PEFT |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-14B", torch_dtype="auto", device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "atlas-institute/qwen3-14b-orchestrator-farca")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-14B")
messages = [
{"role": "system", "content": "You are Nexus, an offensive security orchestrator agent."},
{"role": "user", "content": "Perform a full recon on the target network 10.10.10.0/24 and identify high-value targets."},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=768)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Related Models
| Stage | Model |
|---|---|
| DAPT | atlas-institute/qwen3-14b-dapt-offsec |
| SFT | atlas-institute/qwen3-14b-orchestrator-sft |
| FARCA-GRPO | atlas-institute/qwen3-14b-orchestrator-farca (this model) |
| DPO | atlas-institute/qwen3-14b-orchestrator-dpo |
| GGUF | atlas-institute/qwen3-14b-orchestrator-gguf |
| Abliterated GGUF | atlas-institute/qwen3-14b-orchestrator-abliterated-gguf |
Citations
GRPO method:
@article{shao2024deepseekmath,
title = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
author = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
year = 2024,
eprint = {arXiv:2402.03300},
}
Developed by
Atlas Institute · Part of the Code-Trainer (RTPI) project.