A specialized fine-tuned adapter for Qwen-2.5-3B-Instruct designed to act as a deterministic symbolic rewrite and logic execution engine.
This model does not possess human intuition, nor does it "think abstractly."
Large language models operating in zero-shot conversational mode frequently "fake" mathematical proofs by generating plausible-sounding LaTeX and rhetorical arguments that violate formal inference rules (e.g., asserting $a * b = a$ or claiming $P \to \text{False}$ leaves $P$ undetermined).
This adapter reprograms the 3B model into a strict symbolic transformation engine. It enforces an invariant premise-declaration protocol inside an explicit scratchpad (<think> ... </think>):
If the engine fails to identify a relevant theorem, it experiences a knowledge limit, but it will not violate the structural grammar of formal logic.
The adapter was benchmarked against the raw baseline (Qwen-2.5-3B-Instruct) under low-temperature deterministic sampling ($T \le 0.2$).
Tests used arbitrary or nonsense variables to eliminate world-knowledge bias and test pure deductive mechanics:
| Test Challenge | Base Model (3B) | Logic Engine LoRA (3B) | Mechanistic Breakdown |
|---|---|---|---|
| Categorical Syllogism (Nonsense terms: Zorgs, Vlorps, Plinks) | ❌ Failed | ✅ Passed | Base confused existential deduction ("Some Glips are not Plinks") with universal negation, concluding it was unprovable. The Logic Engine tracked the $\exists x (G(x) \land Z(x))$ witness directly to $\neg P(x)$. |
| Propositional Chain (Modus Tollens: $P \to Q, \neg R \to \neg Q, S \to \neg R, S$) | ❌ Failed | ✅ Passed | Base suffered complete logic breakdown: Claimed $P \to \text{False}$ is "always true regardless of $P$" so $P$ is undetermined. The Logic Engine executed Modus Tollens: $\text{True} \to \text{False}$ is invalid, therefore $P = \text{False}$. |
| Discrete Invariant (Parity Sequence Alternation) | ✅ Passed | ✅ Passed | Both correctly derived that $e$ is even. LoRA adhered to a step-by-step induction invariant. |
| Quantifier Negation (First-Order Predicate Logic) | ✅ Passed | ✅ Passed | Both converted $\neg \forall x (O(x) \to B(x))$ to $\exists x (O(x) \land \neg B(x))$. |
| Total Accuracy | 2 / 4 (50%) | 4 / 4 (100%) | +50% pure deductive improvement over base model |
When evaluating formal mathematical proofs, the contrast between rhetorical text generation (Base) and symbolic rule execution (LoRA) was absolute:
Problem: Prove that if $f(A \cap B) = f(A) \cap f(B)$ for all subsets $A, B \subseteq X$, then $f$ must be injective.
"If $f(y) \notin f({x_1})$, it implies $f({x_1}) \cap f({y}) = \emptyset$, which contradicts the assumption that $f(x_1) = f(x_2)$ should imply some intersection involving other points as well."
(Meaningless mathematical word salad).
Problem: Formally prove $a \mid bc \land \gcd(a, b) = 1 \implies a \mid c$ using Bézout's Identity ($ax + by = 1$).
from unsloth import FastLanguageModel
import torch
model_id = "JPQ24/qwen-2.5-3b-lora-reasoning_logic" # Update with your HF repo
model, tokenizer = FastLanguageModel.from_pretrained(
model_name=model_id,
max_seq_length=2048,
dtype=None,
load_in_4bit=True,
)
FastLanguageModel.for_inference(model)
prompt = """Premises:
1. All Alpha elements are Beta elements.
2. No Beta element is a Gamma element.
3. Some Delta elements are Alpha elements.
Question: Does it logically follow that 'Some Delta elements are not Gamma elements'? Formally prove or disprove."""
messages = [{"role": "user", "content": prompt}]
formatted_prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
# Prime the logic engine
formatted_prompt += "<think>\n"
inputs = tokenizer(formatted_prompt, return_tensors="pt").to("cuda")
outputs = model.generate(
**inputs,
max_new_tokens=1024,
temperature=0.15, # Near-greedy sampling for strict symbolic deduction
repetition_penalty=1.15, # Prevents phrase cycling in derivations
use_cache=True,
)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
transformers + peftimport torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_name = "unsloth/Qwen2.5-3B-Instruct"
adapter_id = "JPQ24/qwen-2.5-3b-lora-reasoning_logic"
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_name,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# Load Logic Engine LoRA
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()
prompt = "Let G be a group where x * x = e for all x in G. Prove formally that G is abelian."
messages = [{"role": "user", "content": prompt}]
formatted_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + "<think>\n"
inputs = tokenizer(formatted_prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=1024,
temperature=0.15,
repetition_penalty=1.15,
)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
Because this model acts as a symbolic re-writer, stochastic sampling introduces logic drift:
0.1 – 0.2 (Near-deterministic; higher temperatures degrade logical consistency)0.91.15 (Essential to prevent repeating formal transformation steps)1024<think>\n after add_generation_prompt=True to immediately engage the Axiom Declaration Phase.This qwen2 model was trained 2x faster with Unsloth
unsloth/Qwen2.5-3B-Instruct