Qwen3.5-4B-typed-decisions-merged

A decision model fine-tuned from unsloth/Qwen3.5-4B with Unsloth on LocalLLaMA/typed-decisions. Instead of generating text, it answers typed questions about an input and returns a probability for each answer in a single forward pass.

This repository contains the full 16-bit model with the LoRA adapters merged in, plus the decision head.

The LoRA adapter version is at wozniakos10/Qwen3.5-4B-typed-decisions.

Question types

Type	Answers	Output
`noul`	A yes/no question	Probability that the answer is yes
`choice`	One option out of named criteria	Probability per option
`score`	A position on an ordered scale	Probability per scale level

Usage

Install Unsloth, then:

from unsloth import FastDecisionModel

model, tokenizer = FastDecisionModel.from_pretrained("wozniakos10/Qwen3.5-4B-typed-decisions-merged", load_in_4bit = True)
FastDecisionModel.for_inference(model)

answers = FastDecisionModel.predict(
    model,
    tokenizer,
    "Hi, I was charged twice for invoice #4411. Please refund the duplicate today.",
    {
        "team": {
            "type": "choice",
            "instructions": "Which team should handle this?",
            "criteria": {
                "billing": "invoices, payments, refunds",
                "technical": "bugs, outages, errors",
                "sales": "pricing, new plans",
            },
        },
        "refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"},
        "urgency": {
            "type": "score",
            "instructions": "How urgent is this?",
            "criteria": ["not urgent", "soon", "today"],
        },
    },
)
for name, result in answers.items():
    print(name, result["answer"], result["probabilities"])

Intended use

Routing, triage, tagging, filtering and other places where code needs a structured judgment with a probability instead of free text. Questions and criteria are supplied at inference time, so no retraining is needed for new labels.

Training

Trained with DecisionTrainer on the all config, train split of LocalLLaMA/typed-decisions. A part of the training rows was held out with FastDecisionModel.split_holdout. The output probabilities have not been temperature-calibrated.

Hyperparameter	Value
Training steps	240
Epochs	1.71
Learning rate (LoRA adapters)	2e-5
Learning rate (decision head)	1e-4
LR scheduler	cosine
Warmup steps	10
Weight decay	0.01
Per-device batch size	2
Gradient accumulation steps	4
Max sequence length	2048
Seed	3407
LoRA rank (r)	16
LoRA alpha	16
LoRA dropout	0

Metric	Value
Final training loss	0.7875
Held-out loss at last evaluation	0.8747

Architecture

Component	Value
Base model	`unsloth/Qwen3.5-4B`
Decision layout	`clef`
Decision head hidden_size	2560
Decision head width	512
Decision head routing_layers	2
Decision head layers	4
Decision head heads	8
Decision head feedforward	2048

Evaluation

Scored with the conventions of the LocalLLaMA/typed-decisions benchmark on the all config, test split (400 cases, 2,000 decisions): one request per case with all five questions. The model was fine-tuned on the train split, so these results are not zero-shot. Measured on the LoRA adapter version (wozniakos10/Qwen3.5-4B-typed-decisions), not re-run on the merged weights.

Metric	Value
Accuracy	0.7725
KL from gold	0.0992
Brier	0.0543
ECE	0.1387

Per-decision predictions are in eval_predictions.jsonl.

Limitations

Probabilities are not temperature-calibrated, so validate on your own data before relying on thresholds.
Trained on English data. Other languages are untested.
Inputs longer than 2048 tokens are outside the trained range.
Outputs are judgments, not facts. Keep a human in the loop for high-stakes decisions.

License

Apache 2.0, inherited from the base model. Check the terms of LocalLLaMA/typed-decisions before commercial use.

Acknowledgements

Trained with the Unsloth decision model guide.

Loading…