Qwen3.5-2B AA Decision — GGUF

GGUF builds of Qwen3.5-2B-AA-Decision, a native decision model that answers typed questions (choice / score / noul) over a state through llama.cpp's TypeSafe-compatible /v1/systemone API — no text generation, one forward pass.

Trained on a mix of LocalLLaMA/typed-decisions and fastino/fast-decisions.

Smaller variant: Qwen3.5-0.8B-AA-Decision-GGUF (much smaller and faster, with a modest accuracy trade-off).

Files

The decision metadata (clef.decision.*) and the systemone chat template are embedded, so llama.cpp detects the model automatically (decision model type: clef).

Usage

llama-server \
  -m Qwen3.5-2B-AA-Decision-Q4_K_M.gguf \
  -c 8192 -b 4096 -ub 2048 -ngl 99 \
  --host 127.0.0.1 --port 8091

Keep -ub 2048. The decision path caps the physical batch; prompts with many options (e.g. 77-way classification) are rejected at the default 512.

curl http://127.0.0.1:8091/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Customer: I was charged twice for my order last week and nobody has replied.",
    "questions": {
      "route":   {"type": "choice", "instructions": "Which team should handle this?",
                  "criteria": {"billing": null, "shipping": null, "technical": null}},
      "angry":   {"type": "noul",   "instructions": "Is the customer angry?"},
      "urgency": {"type": "score",  "instructions": "How urgent is this?",
                  "criteria": ["low", "medium", "high"]}
    }
  }'

Images: pass files: ["data:image/png;base64,..."] (also needs the mmproj from the base Qwen3.5-2B GGUF).

Benchmarks

Raw accuracy, with chance-corrected skill in parentheses. ¹ d1-3B BANKING77 excluded.

License

Apache-2.0 (inherited from Qwen/Qwen3.5-2B and the training data licenses).