Underdog Poodle 4B

Underdog Poodle 4B is Qwen3.5-4B with post training for the agent work Underdog does: calling tools, deciding when not to call one, following instructions, and talking a user through a task. It runs with thinking on. The full write-up is at underdog.ai/poodle.

Against Qwen3.5-4B measured in the same harness with the same settings, it scores higher on every benchmark we track: BFCL-V4 (65.3 vs 56.8), BFCL-120 (80.6 vs 77.5), IFEval (85.4 vs 60.6), IFBench (38.7 vs 33.3), parallel calls (77 vs 69), WebWalkerQA (31.1 vs 30.4) and τ³-bench (60.7 vs 57.9 averaged over four domains).

Evaluation

All numbers below come from one harness, run on Qwen3.5-4B and on this model with identical settings: thinking on, temperature 0.6, top_p 0.95, top_k 20, no presence penalty, and at most 32,768 output tokens. An answer that runs past the budget counts as wrong. We report these settings instead of Qwen's card settings (temperature 1.0 with a presence penalty of 1.5), so the numbers here are not comparable with Qwen's published figures.

BFCL-V4 by category:

τ³-bench runs 2 trials per task with a 131,072-token context. The user simulator and the assertion judge are GPT-5.4 mini, the τ³ package defaults. An episode that fails to produce an official reward counts as a failure for both models.

BFCL-V4 uses the official prompts, checkers and score sheet. The one change is the web-search backend: searches go through OpenRouter's web plugin instead of SerpAPI, and every result is cached so both models saw the same pages.

The model also finishes far more of its answers inside the token budget. On IFEval, 98% of its answers finish against 63% for Qwen3.5-4B; on IFBench, 79% against 43%.

Use

Tested with vLLM 0.32.0 (Python 3.12, NVIDIA H100). The model uses Qwen3.5's chat template and tool-call format:

pip install vllm==0.32.0
vllm serve ConwayResearch/Underdog-Poodle-4B \
  --tool-call-parser qwen3_coder --reasoning-parser qwen3 --enable-auto-tool-choice

Keep thinking on (the default) and use temperature 0.6, top_p 0.95, top_k 20:

curl -s http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "ConwayResearch/Underdog-Poodle-4B",
  "messages": [{"role": "user", "content": "Where is my order 48213?"}],
  "tools": [{"type": "function", "function": {
    "name": "get_order_status",
    "description": "Look up the shipping status of an order.",
    "parameters": {"type": "object", "properties": {"order_id": {"type": "string"}}, "required": ["order_id"]}
  }}],
  "temperature": 0.6, "top_p": 0.95, "top_k": 20
}'

License

CC BY-NC 4.0 (see LICENSE). You can use, share and adapt it for non-commercial purposes with credit to Underdog. Commercial use needs a separate license: write to license@conway.tech.

Built on Qwen3.5-4B by the Qwen team (Apache 2.0; see LICENSE-APACHE and NOTICE). Post training by ConwayResearch.