中文Sign up

distil-labs/distil-qwen3-1.7b-customer-support-deferral-gguf

🤗 Hugging Face sourcetext-generationapache-2.01.7B activated8.0 GBGGUF✓ 1 checksumupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo distil-labs/distil-qwen3-1.7b-customer-support-deferral-gguf ./model-folder
Needs a seeder →

Distil-Qwen3-1.7B-Customer-Support-Deferral (GGUF)

GGUF build of distil-labs/distil-qwen3-1.7b-customer-support-deferral, for serving with llama.cpp.

A fine-tuned Qwen3-1.7B model for multi-turn airline customer support that runs as the small tier of a two-model cascade: it handles most support turns itself and defers genuinely-hard turns to a larger model by emitting a defer_to_larger_model tool call. Every assistant action is a single tool call, including talking to the customer via respond_to_user, so a thin orchestrator can drive it.

Results

Evaluated on a held-out set of airline customer-support turns, scored by an independent GLM-5 judge (score = fraction of responses rated correct).

System Quality Frontier-model calls
Frontier model alone (GLM-5) 0.80 100%
This model + escalation (local) ~0.75 ~4%
Untrained Qwen3-1.7B 0.42 0%

Fine-tuning lifts the local 1.7B from 0.42 to ~0.75 (closing roughly 85% of the gap to its frontier-scale teacher), while running ~96% of turns locally and escalating only the hardest ~4% to the larger model. The escalation is a cost/safety mechanism, not a quality boost over the small model alone.

Usage (llama.cpp)

hf download distil-labs/distil-qwen3-1.7b-customer-support-deferral-gguf \
    distil-qwen3-1.7b-customer-support-deferral.gguf --local-dir models

llama-server \
    --model models/distil-qwen3-1.7b-customer-support-deferral.gguf \
    --port 8000 \
    --jinja

Then query the OpenAI-compatible API at http://127.0.0.1:8000/v1. The airline policy (system prompt) and the 16 tool schemas ship with the demo app as job_description.json.

Demo App

This model powers the Flexible Customer Support Bot demo, a terminal cascade where this local SLM handles most airline-support turns and defers hard turns to a larger, OpenAI-compatible model.

File

File Format Notes
distil-qwen3-1.7b-customer-support-deferral.gguf BF16 Full-precision build (16 bpw). Quantize with llama-quantize if you need a smaller footprint.

Links

License

Released under the Apache 2.0 license. See the transformers model card for base-model and teacher-model license terms.