Qwen3.5-2B AA Decision

A native decision model fine-tuned from Qwen/Qwen3.5-2B with unsloth FastDecisionModel. It answers typed questions (choice / score / noul) over a state through the TypeSafe-compatible /v1/systemone API — no text generation, one forward pass.

This release adds relational supervision (WinoGrande and WANLI) to the previous typed-decisions + fast-decisions mix. WinoGrande +8.4 pts and ANLI +10.7 pts over the previous version, with ARC flat and BANKING77 within ~1.4 pts.

Training data

dataset	license	rows	decisions	role
LocalLLaMA/typed-decisions	Apache-2.0	1,200	6,000	choice/score/noul schema, calibration
fastino/fast-decisions	Apache-2.0	1,445	2,224	routing, intents, multi-domain
allenai/winogrande (`winogrande_debiased`, train)	Apache-2.0	1,500	1,500	coreference
alisawuffles/WANLI (train)	CC-BY-4.0	900	900	NLI / entailment

5,045 rows / 10,624 decisions. The relational block is 22.6% of decisions, class-balanced (750/750 WinoGrande, 300/300/300 WANLI). Only official train splits are used — no test split is trained on. ANLI is used for evaluation only (CC-BY-NC-4.0).

Recipe: QLoRA 4-bit, LoRA r=16 / alpha=16, 3 epochs, lr 1e-4 cosine, effective batch 8, max_len 2048, seed 3407. The published checkpoint is epoch 2.

Benchmarks

Full test splits, served as Q4_K_M GGUF through llama.cpp /v1/systemone (build b11899). Chance-corrected skill = clip((raw - reference) / (1 - reference)).

Benchmark (n)	this model	previous version	Δ
WinoGrande (1267)	0.6219 (0.2439)	0.5383 (0.0766)	+8.4
ARC-Challenge (1172)	0.7372 (0.6496)	0.7381 (0.6507)	−0.1
ARC-Easy (2376)	0.8822 (0.8429)	0.8775 (0.8367)	+0.5
BANKING77 (3076)	0.6001 (0.5949)	0.6141 (0.6090)	−1.4
ANLI (1000)	0.5420 (0.3130)	0.4350 (0.1525)	+10.7

Held-out: typed-decisions test 0.8000, fast-decisions holdout 0.6782.

Calibration

Per-type temperature scaling fitted on a mixed held-out set (unseen WinoGrande/WANLI rows, the fast holdout, and a 240-row typed-train slice) — never on any test split.

ECE	this model
WinoGrande	0.075
ANLI	0.055
typed-decisions test	0.359

One global choice temperature cannot suit both distributions: typed-decisions is under-confident while WinoGrande/ANLI are over-confident, so the fit is a compromise that favours the benchmark tasks. Accuracy is unaffected by calibration.

Version history

ref	contents
`main` (this card)	typed + fast + WinoGrande + WANLI
`v1`	typed + fast (ANLI 0.435)

Load the previous version with revision="v1".

Try the smaller model

The 0.8B sibling, trained on the same mix: Qwen3.5-0.8B-AA-Decision — much smaller and faster, with a modest accuracy trade-off.

Usage

GGUF — llama.cpp `/v1/systemone`

llama-server \
  -m Qwen3.5-2B-AA-Decision-Q4_K_M.gguf \
  -c 8192 -b 4096 -ub 2048 -ngl 99 \
  --host 127.0.0.1 --port 8091

Keep -ub 2048. The decision path caps the physical batch; prompts with many options (e.g. 77-way classification) are rejected at the default 512.

curl http://127.0.0.1:8091/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Customer: I was charged twice for my order last week and nobody has replied.",
    "questions": {
      "route":   {"type": "choice", "instructions": "Which team should handle this?",
                  "criteria": {"billing": null, "shipping": null, "technical": null}},
      "angry":   {"type": "noul",   "instructions": "Is the customer angry?"},
      "urgency": {"type": "score",  "instructions": "How urgent is this?",
                  "criteria": ["low", "medium", "high"]}
    }
  }'

Safetensors

from unsloth import FastDecisionModel
model, tok = FastDecisionModel.from_pretrained(
    "analogalok/Qwen3.5-2B-AA-Decision", max_seq_length=2048)

Files

file	description
`model.safetensors*`	merged 16-bit weights + decision head
`joint_head.safetensors`	decision head weights
`adapter/`	LoRA adapter + head used for the fine-tune
`Qwen3.5-2B-AA-Decision-Q4_K_M.gguf`	~1.3 GB, recommended
`Qwen3.5-2B-AA-Decision-F16.gguf`	~3.6 GB

Limitations

Still below encoder-based decision models on adversarial NLI (ANLI 0.54), and BANKING77 slipped ~1.4 pts versus the previous version. For production routing prefer the Q4_K_M GGUF.

License

Apache-2.0 (inherited from Qwen/Qwen3.5-2B). Training data is Apache-2.0 / CC-BY-4.0; ANLI is used for evaluation only and is not trained on.

analogalok/Qwen3.5-2B-AA-Decision