sqlforge — GRPO on Qwen3.5-4B for multi-step analytical SQL: a negative result, with the artifacts

This repository is a negative result. Two reinforcement-learning climbs and one ablation on Qwen3.5-4B did not beat the base model on the target benchmark (Spider 2.0-Lite, SQLite slice, 135 tasks). The adapters, the per-task judge outputs, the training logs and the report are here so the result can be checked and the failure reused. Code and method: github.com/NakliTechie/sqlforge.

What was tested

Thesis: a 4B model post-trained purely by RL against a deterministic verifier, on tasks at its own learnability frontier, reaches large-model quality on multi-step analytical SQL (question → explore with run_sql → submit one final SQL). Pre-registered criterion: beat the base on the Spider 2.0-Lite SQLite slice, paired per task, 30-database cluster-bootstrap 95 % CI excluding zero, Δ ≥ +5 points. Analysis code: lab/judge_stats.py in the repo.

Results (exec accuracy, Spider 2.0-Lite SQLite slice, 135 tasks)

In-family secondary, BIRD Mini-Dev (496 tasks): climb 2 step150 0.595 vs base 0.514, Δ +8.10 [+4.92, +11.64].

Three findings:

1. The judge harness costs the base 10 points. The same weights score 0.269 under the trainer's rollout path (prior thinking kept in context, top_p 0.95, merged tool messages) and 0.169 under a server-style harness that drops prior reasoning. Judge in the harness you train in.
2. Climb 2 made the policy worse on the target in its own harness while gaining 8 points in-family. On tasks the base could already solve it lost 27 points.
3. The cause is the pool, not the reward. Removing the no-submit penalty (ablation 1, 40 steps) reproduced the loss (base-reachable stratum −18.6). Forty steps on a 90 % BIRD-train pool displace the base's analytical-schema competence.

Files

adapters/climb1/step{20,40,60,80,100}/      LoRA r32 (PEFT), synthetic hop-3/4 pool, 100 steps
adapters/climb2/step{20,40,...,140,150}/    LoRA r32 (PEFT), 521-task pool (BIRD-train + TPC-DS + TPC-H), 150 steps
adapters/ablate1/step{5,...,40}/            climb-2 recipe with no-submit reward 0, 40 steps
results/spider2-eval/                       climb-1 judge: per-task jsonl + summaries (lab.run harness); per-episode
                                            transcripts (messages, thinking, SQL) in each run's transcripts.tar.gz
results/spider2-eval2/                      climb-2 judge: base × 8 seeds, every adapter, BIRD Mini-Dev
results/spider2-eval2b/                     5-seed checkpoint sweep + in-process diagnostic (base and step150)
results/ablate1/                            ablation training log + in-process judge
results/passk*/                             base pass@8 measurements (Spider, TPC-H, TPC-DS, BIRD-train candidates)
results/climb1/, results/climb2/            training-step logs and steering evals
report/                                     the write-up, the four cold reviews + response, climb2_pool.json,
                                            the authored TPC-DS questions and the TPC-H/TPC-DS task files

Every adapter loads with PEFT on Qwen/Qwen3.5-4B (bf16). adapter_config.json sits beside each adapter_model.safetensors. The trainer's vLLM path expects the remapped layout produced by train.vllm_policy.export_adapter.

Reproduce a judge number

git clone https://github.com/NakliTechie/sqlforge && cd sqlforge && uv sync
python -m lab.judge_stats --tasks lab/spider2_sqlite.json \
  --base results/spider2-eval2b/spider2-inprocess-base.jsonl \
  --treat results/spider2-eval2b/spider2-inprocess-adapter.jsonl     # → Δ −6.91, CI [−13.06, −1.56]

Provenance and cost

Spot RTX PRO 6000 on GCP, 39 VM lives, 57.5 GPU-hours, $101.73 total (ledger in the repo's report). Four cold reviews (codex, DeepSeek reasoner, Claude Opus 5.5, opencode/space-bunny) before the judge ran; their forecast, in-family gain that does not transfer, held. Data credits: BIRD (CC BY-SA 4.0), Spider 2.0-Lite (MIT), TPC-H/TPC-DS generated at scale 0.1 with questions authored in this project. Adapters and results: MIT.