Qwen3.5-0.8B complaint-routing decision model (MLX)

A System 1 decision model: it reads a consumer complaint and a typed question with its allowed options once (one forward pass, nothing generated) and returns the chosen financial product with a probability for every option. Built on Qwen/Qwen3.5-0.8B with a LoRA adapter and a small joint schema head, trained and run with Apple MLX.

Training status: completed.

Model overview

Model details

• Base model: Qwen/Qwen3.5-0.8B at commit 2fc06364715b967f1860aea9cf38778875588b17, license apache-2.0. MLX-LM loads its text model; the vision tower is not used.
• This repository is not a standalone model. It holds a LoRA adapter (adapters.safetensors, adapter_config.json, MLX-LM format), the head (head.safetensors, head_config.json) and the self-contained loader decision_model_runtime.py. All three are applied on top of the exact base model above.
• Head: {"hidden_size": 1024, "width": 512, "routing_layers": 1, "layers": 2, "heads": 8, "feedforward": 2048}.
• Trainable parameters: LoRA 10,223,616, head 15,763,460 (backbone 762,616,640).
• Weight format: safetensors (MLX); base weights bf16, head float32.
• Software: mlx 0.32.4, mlx-lm 0.32.2.

Intended use

Routing short English consumer-finance complaints to the product team that handles them, with calibrated-looking probabilities that a workflow can threshold, for experiments and teaching about decision models versus generative answers. The request may offer any subset (at least two) of the 9 routes; the model only chooses among the offered ones.

Not for: decisions about individuals (credit, collections, eligibility), other domains or label sets without retraining, or unattended routing where a wrong route harms a consumer. Keep a human review path for low-confidence answers.

Training data

• Dataset: kgptalkie/cfpb-complaint-routing, revision 3ec70ba3690c26b8d9b117629fa3803ce4e4b84f, license cc0-1.0: short complaints from the US CFPB Consumer Complaint Database (public data; personal details redacted as XXXX by the CFPB), labelled with 9 products, balanced train split.
• Rows: train 2700, validation 270, test 826.
• Training stream: 800 requests drawn without replacement from the train split with seed 7; 46.9% of them offer only a random subset of the routes (always including the right one), so the head has to read the options instead of memorising 9 fixed slots.
• Prompt layout: a system line, STATE: with the request state as compact JSON, then SCHEMA FIELDS: with each question's id, type, instruction and every allowed option as {"option_id": …, "description": …}, then an empty think block and JOINT SCHEMA DECISIONS:. System line:

Read the complete state and schema. Decide every field jointly. Each answer must be exactly one of that field's allowed options.

• Question: Which financial product is the consumer's complaint about?
• Target: the index of the correct option; loss is cross-entropy over the offered options.
• Overlap: no id or complaint text is shared between train, validation and test (train/validation {'ids': 0, 'texts': 0}, train/test {'ids': 0, 'texts': 0}, validation/test {'ids': 0, 'texts': 0}).
• Limitations: short complaints only, US consumer-finance vocabulary, CFPB product taxonomy mapped to 9 routes; labels reflect what consumers selected when filing, which is sometimes debatable.

Training procedure

Evaluation

The 826 test complaints of the dataset, never used for training, each offered all 9 routes.

• Decision model (System 1): one forward pass; the answer is the option with the highest logit. Reported with expected calibration error of its confidences.
• Chat baseline (System 2): the base model asked the same request as a chat (greedy, thinking off, at most 16 new tokens); the answer is the product of the last JSON object in the reply. A reply without a valid route counts as wrong.

Accuracy is measured against the dataset's labels; a 95% interval at this size is up to about ±3 percentage points. Latency is per complaint on the machine that ran the evaluation.

How to use

Requires macOS on Apple Silicon.

pip install "mlx-lm>=0.32.2,<0.33" huggingface_hub

import json, sys
from huggingface_hub import snapshot_download

folder = snapshot_download("thealper2/mlx-qwen3.5-0.8b-decision-model")
sys.path.insert(0, folder)
from decision_model_runtime import decide, load_release

model, tokenizer = load_release("Qwen/Qwen3.5-0.8B", "2fc06364715b967f1860aea9cf38778875588b17", folder)
request = {
    "state": {"complaint": "I wired $1,200 to a contractor; it left my account "
                            "but never arrived and the bank will not trace it."},
    "questions": {
        "product": {
            "type": "choice",
            "instructions": "Which financial product is the consumer's `complaint` about?",
            "criteria": {
                "credit_reporting": "credit reports, credit bureaus, credit repair, identity on report",
                "debt_collection": "debt collectors, collection calls, debts owed",
                "credit_card": "credit cards and prepaid cards",
                "bank_account": "checking or savings accounts",
                "mortgage": "home loans and mortgages",
                "money_transfer": "money transfers, wire, payment apps, virtual currency",
                "vehicle_loan": "car loans or vehicle leases",
                "student_loan": "student loans",
                "personal_loan": "payday, title, personal or installment loans",
            },
        }
    },
}
print(json.dumps(decide(model, tokenizer, request), indent=2))

The response holds answers.product.choice, its confidence and the probability of every offered option; usage.output_tokens is 0 because nothing is generated. To restrict the choice, pass fewer criteria (at least two).

Limitations and risks

• Misroutes complaints that mention several products (a credit-card dispute that is also on a credit report) and complaints whose label is itself debatable.
• Probabilities are a model output, not guarantees; check calibration on your own data before thresholding.
• Trained on short English US complaints and these 9 routes; other wording, lengths, languages or taxonomies are out of distribution.
• Only choice questions are supported by the published runtime.
• The head was trained jointly with this adapter; neither works with another base model or revision.
• Small model on consumer hardware: no production-readiness guarantees.
• Do not use it to make or justify decisions about individual consumers.

License and attribution

• Weights in this repository: released under apache-2.0 (as configured by the publisher).
• Base model: Qwen/Qwen3.5-0.8B, license apache-2.0.
• Data: kgptalkie/cfpb-complaint-routing, license cc0-1.0; derived from the US CFPB Consumer Complaint Database (public data).
• Head design: follows Cloudflare's Clef joint schema model (Apache-2.0); this is an independent MLX implementation and is not affiliated with or endorsed by Cloudflare.
• The code, data and model licenses are independent; check each before reuse.

Reproducibility

The project that produced this model keeps every setting in configs/ and drives everything through make:

make install
make data-prepare        # splits + the ordered training-request stream
make data-validate
make evaluate-chat       # System 2 baseline
make train PROFILE=m1_air_8gb
make evaluate            # System 1 decision model
make card
make publish-dry-run

Artifacts: run directory artifacts/runs/m1_air_8gb with run_manifest.json (configuration, versions, loss per update, timings), evaluation records under reports/evaluation/. Hardware: an Apple Silicon Mac; the m1_air_8gb profile targets an 8 GB M1 MacBook Air.