TinyQuery-140M

GitHub: source code, setup guide, streaming examples and tests

139.7M parameters, trained from random initialization on one RTX PRO 6000 within a four-hour experiment. TinyQuery turns a question, database schema and runtime tool definitions into a JSON action. It targets English, imperfect English, Hindi and Hinglish, with bounded read-only MySQL and PostgreSQL/Supabase queries and MCP-style tool calls.

The frozen student achieved 92.47% full task success (1,106/1,196) on the held-out test and 82.50% (132/160) on additional handwritten-phrasing cases. It is an experimental narrow model, not a general chatbot or a guarantee of correct SQL.

This repository contains the weights, training/inference source, tokenizer, configurations, evaluation reports, architecture documentation and runtime recipes. This is a custom native PyTorch checkpoint. Transformers AutoModel and pipeline() cannot load it.

Run with streaming output

Requires Python 3.12. Download this repository, then run from its directory:

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python -m tinyquery.chat "दिल्ली के ग्राहकों के नाम दिखाओ।" \
  --checkpoint model.safetensors --backend supabase --mcp --stats

The CLI automatically uses Apple Silicon MPS, CUDA or CPU and streams raw output. The bundled fictitious demo schema supports this example:

{"action":"call","name":"execute_sql","arguments":{"query":"SELECT name FROM customers WHERE city = 'Delhi';"}}

Use --context examples/context-supabase.json or examples/context-mysql.json, edited with your schema and exact tool definitions. A context has backend, project_id, schema, tools and policy; tools use name, description and JSON Schema inputSchema. The context limit is 2,048 tokens including the output budget; final training sequences reached 945 tokens.

--mcp validates and serializes a JSON-RPC tools/call payload. It does not connect to an MCP server, authenticate, or execute a database query. The host application must implement transport and execution. The structural validator does not prove semantic correctness. Invalid output causes a nonzero CLI exit status.

A single local M5/MPS demonstration measured approximately 52 generated tokens/second, excluding model loading. This is one short run, not a throughput guarantee.

Evaluation

Weights were frozen at step 25,715 on 2026-09-10 at 14:00:36 UTC, before reading test/manual results. Selection maximized the equal mean of full validation and development success among completed evaluated candidates. Development contains schema-layout and tool-name perturbations of validation families.

These primary numbers are the completed CUDA BF16 evaluation. Complete raw GPU predictions and summaries are in evaluation/test-gpu.* and evaluation/manual-gpu.*. Independent FP32 Mac evaluation reproduced every aggregate metric exactly; complete raw predictions, language/backend breakdowns and summaries are included in evaluation/test-mac.* and evaluation/manual-mac.*. No retraining or selection follows the test results.

Evaluation uses raw greedy generation with no repair, constrained decoding, retrieval fallback or teacher fallback. Full SQL-task success requires the correct tool, exact non-SQL arguments (including project scope), compilation against the supplied schema, and matching results on two generated SQLite fixtures after dialect adaptation. SQL equivalence alone does not require correct project scope, so its percentage can exceed full success. Non-SQL tool calls require exact arguments; clarification/answer text requires exact reference wording.

These are synthetic benchmark results, not measured production task success. Native MySQL 8.0.46/PostgreSQL 16.15 checks passed all 66 reference-operation/dialect cases. Final student checks passed 859/912 SQL tasks and 91/112 manual SQL tasks, requiring correct tool and project scope as well as result equivalence. Combined with non-SQL results, these reproduce exactly the full-task totals of 1,106/1,196 and 132/160. Native reports are included in evaluation/native-*.json. SQL fixture equivalence is not a proof for all possible databases.

Validation scored 1,174/1,200 (97.83%); development scored 188/192 (97.92%). These were used for model selection and are not final held-out claims. The 160 manual cases use handwritten phrasing templates on 40 test families, so manual and test are not independent schema-family samples.

Architecture and training

The source-copy head mixes vocabulary generation with attention over the supplied context and question, excluding generated answer text. It adapts the established pointer-generator approach. The architecture and dataset evolved together, so this is not a controlled ablation or a claim of a novel research architecture. See architecture documentation.

All student weights originate from random initialization during this experiment; the copy head was initialized later and trained with the continuing decoder. No pretrained student checkpoint or teacher logits were used. Training used BF16, AdamW, gradient clipping and weighted length buckets. Early loss weights were response 1, prompt 0.15, and auxiliary action CE 0.05. A later response-only refinement did not displace the selected checkpoint.

The selected checkpoint records 595,067,301 processed tokens, 78,367,410 response tokens and 5,092.34 trainer seconds. These are retained-lineage counters, including validation/checkpoint time and excluding discarded work and model loading. Training continued to step 32,642, but those last weights were not selected. The overall experiment began at 10:57:23 UTC and had a 14:57:23 UTC deadline. See experiment provenance for curricula and counters.

Dataset and teacher

The companion TinyQuery-Tools-Multilingual dataset contains 347,376 final training rows, 25,145 scenario groups, 100,224,712 tokens and 13,163,249 response tokens. Language/context variants are not independent teacher generations. SQL/tool semantics are programmatic; Qwen supplies templates, paraphrases and verification. An additional city-language curriculum is programmatically authored and labeled accordingly.

Teacher: Qwen/Qwen3.8-27B-FP8, revision 017b9c7af6b5689d5dd426a76e0bc077eb5ca20a. On the rented RTX PRO 6000, vLLM 0.28.0 with MTP-3 measured about 931 output tokens/second versus 691 without MTP at 32 concurrent requests. Initial template/paraphrase generation preceded MTP; verification and template audit used MTP. This is a measured workload comparison, not a claim of globally optimal settings. See teacher runtime provenance and runtime recipes.

Training has no scenario-group overlap with validation, development, test or manual. There are no exact prompt duplicates across splits. Development shares validation families, and manual shares test families, intentionally. A teacher audit flagged 37 template instances; 6,778 derived rows were removed from the final corpus after earlier training stages had already encountered them. Final reference-data audits passed schema/serialization and generated-fixture checks; these checks do not certify all natural-language paraphrases.

Scope and limitations

• Supports a bounded recipe set: projections, filters, aggregates, grouping/HAVING, ordering, date/string filtering and a single join, plus schema discovery and a few generic tool categories.
• Arbitrary unseen databases, tools, complex joins, long conversations and broad Hindi translation are not established capabilities. Hindi city/table lexical mappings cover a small explicit training vocabulary.
• A correct JSON object can contain a wrong tool, literal, project ID or query. The model is not 100% reliable.
• It emits one action, not a complete autonomous MCP agent. Database execution and authorization belong to the host application.
• The runtime source and frozen data support new experiments; the release is not a bitwise replay of the evolving training session. Runtime configs, training logs and evaluation records were recovered after a brief SSH outage. The optional final optimizer-state download was stopped to avoid extending rental cost; inference weights are complete.

Weights and project code are released under Apache-2.0. Teacher provenance is documented separately. See manifest.json for file hashes and checkpoint-info.json for selected-weight counters.