A specialized reasoning adapter for Qwen-2.5-3B-Instruct designed for systems architecture, cybernetic feedback analysis, and non-linear dynamics.
Standard instruction-tuned models often fail at systemic reasoning—they hallucinate feedback loop polarities, confuse rates of change with physical accumulations, and miss transport delays. This adapter instills a structured systems thinking cognitive posture, enforcing an explicit scratchpad (<think> ... </think>) that separates analysis into Divergent Mapping (structural isomorphism) and Principled Evaluation (boundaries, trade-offs, and failure modes).
The adapter was benchmarked head-to-head against raw Qwen-2.5-3B-Instruct across classical non-linear dynamics problems.
| Dynamic Challenge | Base Model (3B) | Systems LoRA (3B) | Key Systemic Distinction |
|---|---|---|---|
| 1. Policy Resistance & Causal Loops (Cobra Effect) | ❌ Flawed | ✅ Passed | Base mislabeled a surge of released predators as a "Balancing Loop" and assumed agents work without economic gain. LoRA cleanly mapped 4 distinct balancing/reinforcing loops and dual delays. |
| 2. Stocks, Flows, & Delays (The Shower Problem) | ❌ Hallucinated | ✅ Passed | Base hallucinated a "Temperature Sensitivity Stock". LoRA correctly treated water heat as the stock, valve turn as the flow, and diagnosed the delayed homeostatic feedback loop. |
| 3. Cross-Domain Isomorphism (Immunology $\longleftrightarrow$ Cybersecurity) | ❌ Failed Analogy | ✅ Passed | Base incorrectly mapped Autoimmunity to Insider Threats. LoRA correctly identified autoimmunity as false-positive self-attacks (blocking legitimate users/services). |
Challenge: Analyze the causal loops, information/policy delays, and systemic failure when a bounty on venomous snakes leads to backyard breeding and subsequent release.
Challenge: Model the oscillatory overshoot caused by a 10-second transport delay between turning a shower valve and water reaching the showerhead.
Challenge: Map the human immune system to enterprise security and evaluate the failure mode when the system becomes "autoimmune".
from unsloth import FastLanguageModel
import torch
model_id = "JPQ24/qwen-2.5-3b-lora-reasoning_natural_synthesis_v5"
model, tokenizer = FastLanguageModel.from_pretrained(
model_name=model_id,
max_seq_length=2048,
dtype=None,
load_in_4bit=True,
)
FastLanguageModel.for_inference(model)
prompt = "Analyze the bullwhip effect in supply chains. Map the stocks, flows, and information delays causing inventory oscillations."
messages = [{"role": "user", "content": prompt}]
formatted_prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
# Prime the model to activate the systems scratchpad
formatted_prompt += "<think>\n"
inputs = tokenizer(formatted_prompt, return_tensors="pt").to("cuda")
outputs = model.generate(
**inputs,
max_new_tokens=1400,
temperature=0.25, # Low temperature ensures disciplined loop mapping
repetition_penalty=1.15, # Prevents delimiter looping
use_cache=True,
)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
transformers + peftimport torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_name = "unsloth/Qwen2.5-3B-Instruct"
adapter_id = "JPQ24/qwen-2.5-3b-lora-reasoning_natural_synthesis_v5"
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_name,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# Load Systems LoRA
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()
prompt = "Using systems dynamics, explain why adding more lanes to a highway often increases traffic congestion (Braess's Paradox)."
messages = [{"role": "user", "content": prompt}]
formatted_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + "<think>\n"
inputs = tokenizer(formatted_prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=1400,
temperature=0.25,
repetition_penalty=1.15,
)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
To prevent structural drift and keep the model's feedback loops grounded:
0.2 – 0.30.91.15 (Crucial for stopping template echoes)1200 – 1600 (Gives ample token budget for both the <think> mapping and the systemic solution)<think>\n after add_generation_prompt=True to immediately trigger the Divergent Mapping phase.This qwen2 model was trained 2x faster with Unsloth
unsloth/Qwen2.5-3B-Instruct