askmdb-9b (GGUF)

A LoRA fine-tune of Qwen/Qwen3.5-9B, merged into the base weights, that turns a question about a MongoDB database, in Spanish or English, into one read-only aggregate pipeline, answered as a small JSON object. It reads a compact description of the database's schema in the prompt, so it works on databases it has never seen. It was trained with the same data and contract as askmdb-0.8b. It is the larger GPU model of askmdb, an offline command-line tool that runs it through llama.cpp with Vulkan when you choose it, and it works with llama-server in any other application, on CPU or GPU (family).

• Author: Manuel García Sanz
• Website: https://manuelgs.com
• Source code of askmdb: https://github.com/manuelgscom/askmdb
• Format: GGUF, Q4_K_M (imatrix) · 5,629.1 MB
• Test accuracy: 91.6 % (458/500), scored by execution, versus 47.8 % for Qwen3.5-9B and 45.4 % for Gemma 4 E4B-it without fine-tuning
• In the askmdb CLI: offered on the GPU when a dedicated GPU has at least 8 GiB of free VRAM, if you choose it over askmdb-4b. 1.85 s per question and 5.5 GiB of VRAM (details)
• The askmdb family: four sizes of the same model (0.8B, 2B, 4B, 9B), all in one collection

askmdb live: ask in plain English; it shows the pipeline, explains it, checks that it only reads and runs it (in auto mode, a join asks first). Real, unedited recording with askmdb-0.8b on a desktop CPU. With askmdb-9b on the GPU the flow is the same, with a more accurate model.

Resumen en español

askmdb-9b es un ajuste LoRA de Qwen3.5-9B, fusionado con los pesos del modelo base, que convierte una pregunta en castellano o en inglés sobre una base de datos MongoDB en un único pipeline de aggregate de solo lectura, con una explicación breve. Lee en el prompt una descripción compacta del esquema, así que sirve para bases que nunca ha visto. La CLI askmdb lo ofrece en la GPU (con Vulkan) cuando una GPU dedicada tiene al menos 8 GiB de VRAM libre y el usuario lo elige. También sirve con llama-server en cualquier otra aplicación. En el test público de 500 casos, puntuado ejecutando cada consulta, acierta 458 de 500 (91,6 %), frente al 47,8 % de Qwen3.5-9B sin ajustar. El modelo no decide la seguridad: dentro de askmdb lo hace un verificador determinista y solo se ejecutan pipelines de lectura; fuera de askmdb no hay verificador, así que revisa siempre la consulta y conéctate con un usuario de solo lectura.

The askmdb family

The same task, data, contract and prompt in four sizes. Scores on the public test set manuelgscom/askmdb-test (500 cases, scored by execution), all in Q4_K_M.

Which one?

 the askmdb CLI, first run:
   dedicated GPU, ≥ 8 GiB free VRAM ──► you choose: 1) askmdb-4b (Enter, faster)  2) askmdb-9b (more accurate)
   dedicated GPU, ≥ 4 GiB free VRAM ──► askmdb-4b
   anything else (CPU, integrated GPU) ──► askmdb-0.8b on the CPU

 your own application (llama-server, CPU or GPU) ──► any of the four, askmdb-2b included:
                                                     pick by memory and accuracy

What it is and intended use

 your app ──► schema block ──► askmdb-9b   ──────────► your own checks ──────► run it with a
              (db, collections,  (llama-server, CPU   (outside askmdb there    read-only user
               field paths,       or GPU: one          is no verifier: reject
               types, values)     pipeline, refuse     writes, $out, $merge,
                                  or unsure)           unknown stages…)

• The model proposes; code decides. The model writes one aggregate pipeline (or refuse / unsure). It never decides whether that pipeline is safe to run.
• Honest when it cannot answer. It answers refuse for writes, administration or off-topic requests, and unsure when the data is not in the schema, instead of inventing a query.
• Schema-aware. The prompt carries a compact schema block (collections, field paths, types and frequent values), so the field names and spellings come from your database.
• Same contract as askmdb-0.8b (section below): a drop-in replacement wherever you call that model through llama-server.
• Outside askmdb there is no verifier. Your application must review every pipeline before running it, and should connect with a MongoDB user that only has the read role.

The model was trained only for this contract. It is not a general chat model.

In the askmdb CLI

askmdb is one binary for CPU and GPU. On the first run it looks for a dedicated GPU through Vulkan. When a dedicated GPU has at least 8 GiB of free VRAM it asks between askmdb-4b (Enter, faster) and this model (more accurate). It asks before downloading (5.6 GB, checked with SHA-256), saves the choice (askmdb --detect asks again) and falls back to askmdb-0.8b on the CPU if the GPU check fails. The GPU needs the Vulkan loader (libvulkan1 / vulkan-loader), which almost every desktop already has.

GPU support has been tested on NVIDIA only. Detection and inference use standard Vulkan, but AMD and Intel GPUs have not been tested yet.

Times are for the whole askmdb process per question (start, model load, prompt, generation), as the user perceives it, on one reference desktop with a 16 GB GPU (CPU runs with 8 threads): 30 questions, five runs each with the schema already cached and one first question against the database. They depend on the machine. askmdb starts one process per question and loads the weights each time, which is most of the time on the GPU.

Safety in askmdb: the read-only verifier

Safety in askmdb is decided by code, not by the model: a write request gets an honest "no", and a hand-written $out hidden in a $facet is blocked by the verifier (shown with its standalone tool, askmdb_verify, built from the same source). Real, unedited recording with askmdb-0.8b; the verifier is the same whatever model runs.

• A deterministic verifier in C++ walks the exact BSON that would be sent to the server and gives every pipeline a level: read (known read-only stages and operators), warn (reads, but may be expensive or surprising, such as $lookup or $sample: never runs without an explicit yes) or blocked (writes, administration, JavaScript, other databases, or anything it does not understand: never runs, in any mode).
• The most restrictive part wins, sub-pipelines included ($facet, $lookup, $unionWith). Duplicated keys and multi-key stages are blocked.
• askmdb only runs aggregate, with a time limit and a row limit. It is not a replacement for access control: connect with a user that has the read role.

Files

sha256sum askmdb-9b-1.0.0-Q4_K_M.gguf

The GGUF carries general.* metadata (name, author, URL, version, license, base model). The tensors are identical to the file evaluated below; only the metadata was added.

Using it with llama-server (CPU or GPU)

askmdb builds the prompt itself, so none of this is needed to use the CLI. To call the model from your own code, follow the contract (version 1) exactly; the model was trained on this format and nothing else. Reasoning is always disabled.

Prompt

The prompt is Qwen3.5's chat template with thinking disabled (apply_chat_template(..., add_generation_prompt=True, enable_thinking=False)), with two turns only: a system turn (fixed instructions, a blank line, then the schema block) and the question. No few-shot examples, no reasoning.

Instructions (exactly two lines; $$NOW is literal):

Answer the question with one MongoDB aggregate pipeline over the database described below.
Reply with only JSON {"type":"query|refuse|unsure","collection":"...","pipeline":[...],"explanation":"..."}: type is refuse for writes or off-topic requests and unsure if the schema lacks the data, and then collection is "" and pipeline is []; use $$NOW for relative dates; explanation is one short sentence in the language of the question.

Schema block (an invented example database):

db=bakery
## orders
_id oid
placed_at date
customer_id str
channel str ["shop","web","phone"]
status str ["delivered","pending","cancelled"]
total double
items[] obj
items[].product_id str
items[].qty int
## products
_id str
name str
category str ["bread","pastry","cake"]
price double
gluten_free bool

• First line db=<database>, then one ## <collection> header per collection, in alphabetical order.
• One line per field path: path type [frequent values]. Dotted paths for subdocuments; items[] is an array and items[].qty a field of its elements. A ? after the path marks a field missing in some sampled documents.
• Types: oid str int long double decimal bool date obj arr null bin other; mixed types as unions (int|str).
• Frequent values (a JSON array) only for strings with at most 12 distinct values of at most 30 characters; never values that look like an email or a phone number.
• Limits: depth 4, at most 40 paths per collection, about 4,500 bytes in total (trimmed deterministically: values first, then the deepest paths). In askmdb the block comes from sampling 200 documents per collection; askmdb --print-schema prints it, so you can reuse exactly the block the CLI would send.

User turn: the question, with whitespace collapsed to single spaces, at most 1000 bytes.

Full prompt string (line breaks shown for readability; \n are real newlines):

<|im_start|>system\n{instructions}\n\n{schema block}<|im_end|>\n<|im_start|>user\n{question}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n

The prompt ends with an empty <think> block, so the model starts writing the JSON straight away. The schema block has no trailing newline (the template trims every message). Tokenize with special tokens parsed and without BOS. A context of 4096 tokens is enough for the largest schema block plus the answer.

Answer

Exactly one JSON object with exactly these four keys, in this order:

{"type":"query|refuse|unsure","collection":"...","pipeline":[...],"explanation":"..."}

• type is query, refuse (writes, administration or off-topic) or unsure (the data is not in the schema).
• With query, collection is one collection of the schema and pipeline is an array of 1 to 20 stages, each an object with a single key. Otherwise collection is "" and pipeline is [].
• Special values in relaxed Extended JSON: {"$date":"2026-01-01T00:00:00Z"} (UTC), {"$oid":"…"}, and {"$numberDecimal":"…"} only for decimal fields.
• Relative dates use $$NOW with $dateTrunc / $dateSubtract / $dateAdd inside $expr; MongoDB resolves $$NOW when the pipeline runs.
• The model adds $limit only when the question asks for a top N; askmdb always appends its own row limit.
• explanation is one short sentence in the language of the question.
• Nothing before or after the object. askmdb cuts the output at the closing top-level } and rejects anything else as invalid (it runs nothing).

Decode greedily and cap at 384 tokens (as askmdb does).

Example

llama-server -m askmdb-9b-1.0.0-Q4_K_M.gguf -ngl 99 -c 4096 --reasoning off --port 8080

-ngl 99 puts every layer on the GPU. To run on the CPU only, use -ngl 0 (or --device none) instead: without -ngl, a GPU build of llama.cpp offloads layers on its own (-ngl auto). Any llama.cpp backend works (CPU, CUDA, Vulkan…); the askmdb CLI uses Vulkan. --reasoning off sets enable_thinking=false for the chat API (checked with llama.cpp v0.7.0; the flag exists since build b8287); older builds use --jinja --chat-template-kwargs '{"enable_thinking":false}', which v0.7.0 still accepts with a deprecation warning.

Raw prompt with /completion (recommended: you control the exact string):

import json, urllib.request

INSTRUCTIONS = (
    "Answer the question with one MongoDB aggregate pipeline over the database described below.\n"
    'Reply with only JSON {"type":"query|refuse|unsure","collection":"...","pipeline":[...],"explanation":"..."}: '
    'type is refuse for writes or off-topic requests and unsure if the schema lacks the data, '
    'and then collection is "" and pipeline is []; use $$NOW for relative dates; '
    "explanation is one short sentence in the language of the question."
)
SCHEMA = """db=bakery
## orders
_id oid
placed_at date
customer_id str
channel str ["shop","web","phone"]
status str ["delivered","pending","cancelled"]
total double
items[] obj
items[].product_id str
items[].qty int
## products
_id str
name str
category str ["bread","pastry","cake"]
price double
gluten_free bool"""
question = "Revenue per sales channel in the last 30 days"

system = INSTRUCTIONS + "\n\n" + SCHEMA
prompt = (
    f"<|im_start|>system\n{system}<|im_end|>\n"
    f"<|im_start|>user\n{question}<|im_end|>\n"
    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
)
body = {"prompt": prompt, "n_predict": 384, "temperature": 0, "top_k": 1}
req = urllib.request.Request("http://127.0.0.1:8080/completion",
                             data=json.dumps(body).encode(),
                             headers={"Content-Type": "application/json"})
answer = json.loads(json.load(urllib.request.urlopen(req))["content"])
print(answer["collection"], json.dumps(answer["pipeline"]))

Chat API with /v1/chat/completions: the server applies the chat template stored in the GGUF; turn reasoning off per request with chat_template_kwargs (or with --reasoning off on the server, as above):

body = {
    "messages": [
        {"role": "system", "content": system},
        {"role": "user", "content": question},
    ],
    "chat_template_kwargs": {"enable_thinking": False},
    "max_tokens": 384, "temperature": 0, "top_k": 1,
}
req = urllib.request.Request("http://127.0.0.1:8080/v1/chat/completions",
                             data=json.dumps(body).encode(),
                             headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req))["choices"][0]["message"]["content"])

The chat template stored in this GGUF is the one the model was trained with: Qwen3.5's template, which keeps reasoning off unless enable_thinking is true. llama-server, however, passes enable_thinking: true by default (--reasoning auto): without --reasoning off on the server or enable_thinking: false in the request, the prompt ends with an open <think> block, outside the contract: the JSON may come back in reasoning_content with an empty content, or after a reasoning the model was not trained to write. A request's chat_template_kwargs overrides the server flag. The original Qwen3.5-9B template also turns reasoning on by default, so if you replace the template (for example with --chat-template-file), keep enable_thinking: false too.

Answers of askmdb-0.8b on that schema; this model answers in exactly the same format:

{"type":"query","collection":"orders","pipeline":[{"$match":{"$expr":{"$gte":["$placed_at",{"$dateSubtract":{"startDate":"$$NOW","unit":"day","amount":30}}]}}},{"$group":{"_id":"$channel","total":{"$sum":"$total"}}}],"explanation":"Sums the order totals per sales channel over the last 30 days."}
{"type":"query","collection":"orders","pipeline":[{"$match":{"$expr":{"$gte":["$placed_at",{"$dateTrunc":{"date":"$$NOW","unit":"year"}}]}}},{"$unwind":"$items"},{"$group":{"_id":"$items.product_id","unidades":{"$sum":"$items.qty"}}},{"$sort":{"unidades":-1}},{"$limit":5}],"explanation":"Muestra los 5 productos con más unidades vendidas este año."}
{"type":"refuse","collection":"","pipeline":[],"explanation":"askmdb only reads data: it cannot delete documents."}
{"type":"unsure","collection":"","pipeline":[],"explanation":"The schema has no courier data for orders."}

Never run a pipeline blindly. Outside askmdb there is no verifier: reject anything that is not a plain read ($out, $merge, unknown stages, JavaScript operators such as $function or $where), and connect with a user that only has the read role.

Evaluation

Test set: manuelgscom/askmdb-test, 500 questions on 10 synthetic MongoDB databases (254 in Spanish, 246 in English; 449 with a query, 31 that must be refused, 20 whose data is not in the schema). Every case is scored by execution: the predicted pipeline runs on the same database as the reference and the results are compared; no model grades the answers.

All models in Q4_K_M, reasoning disabled, greedy decoding, same llama.cpp version, 384-token cap. The askmdb models get the plain contract above, through the askmdb binary. The models without fine-tuning get the same instructions plus six worked examples on a separate example database (few-shot), through llama-server, which helps them.

MB = 10^6 bytes. The Qwen3.5 rows are the published Qwen/Qwen3.5-* models; the Gemma 4 models are the instruction-tuned releases, with effective parameters (their files are larger than a Qwen model with the same count).

Where the errors are

Fine-tuning almost removes format errors (invalid JSON, wrong type, blocked pipelines) and the "when not to answer" errors. Most of the remaining errors are a valid, read-only pipeline that runs but computes something else: 34 of this model's 42 wrong answers.

By language: 91.3 % in Spanish (232/254) and 91.9 % in English (226/246).

By category: the hardest categories for this model are joins ($lookup) (22/31) and arrays (41/48). Categories are small (a few dozen cases each), so differences of a few cases are noise.

Per-category table (correct answers)

Methodology

• Independent, synthetic test. 10 databases filled by a seeded generator from declarative specs (published with the test set); half of them come from domains that no training database covers. Each question and its reference pipeline were written by one LLM; an LLM of a different family wrote a second pipeline blind, and a case was kept only when both returned the same result and the reference passed mutation checks (small changes to the pipeline must change the result). The test was frozen with a SHA-256 before any model was evaluated, and the training data was decontaminated against it.
• Scored by execution, lenient on form, strict on content. Column names, key order, extra columns, numeric types, a date or ObjectId written as a string or rounding to 2 decimals do not matter; a different value, a missing or extra row, or a wrong order (when the question asks for one) do. $$NOW is replaced by each database's fixed reference date, so results do not depend on the day of scoring.
• A pipeline the askmdb verifier blocks counts as wrong, even if it would return the right data.
• What it does not measure. The test only keeps questions with one clear answer; real questions can be ambiguous. Verdicts have no manual review. With 500 cases the 95 % interval is about ±3 points, and much wider per category. Each size was trained once (one seed), so neighbouring sizes a few cases apart are not clearly ordered.

Training

Data. 15,000 synthetic examples (14,248 for training, 752 for validation), questions in Spanish and English (about half each) on about 110 generated training databases, none of them a test database. They were written with LLM assistance in three ways: a question and two pipelines from two independent LLM authors, kept only when both return the same result; pipelines built by code from intent templates with real values, phrased as questions by an LLM; and templated refuse / unsure cases. Every example passed the output contract, the askmdb verifier, execution on its database with a non-empty result, mutation checks, near-duplicate removal and decontamination against the test. The schema block in each prompt was produced by the askmdb binary itself. The training data and the training scripts are not published.

Caveats of the 9B.

• Its recipe differs from the other three sizes (LoRA instead of a full fine-tune, and a learning rate that was not tuned), so the size comparison is not like for like.
• One training run (one seed), like the other sizes.
• On the test it gets 458 right against 448 for askmdb-4b: a lead of a few cases, which is within run-to-run noise. Pick it for the extra margin if your GPU has the memory, not for a guaranteed step up.

Limitations

• aggregate only, read only. No find, no writes: write requests are refused by the model and blocked by askmdb. MongoDB 5.0 or newer.
• No Atlas Search or vector search ($search, $vectorSearch).
• It can be wrong: it gets 458 of the 500 test cases right. A wrong pipeline can look plausible and run fine. Always check the result.
• It is not a security tool. The model can propose a pipeline that writes or is expensive; it does not classify risk. In askmdb the verifier does; outside askmdb, your application must.
• It only knows what the schema block shows. askmdb samples 200 documents per collection: rare fields can be missing, and very large schemas are trimmed (at most 40 paths per collection, depth 4).
• Joins ($lookup) are the weakest category for every size.
• Trained for one task with one fixed prompt. Other prompts, chat use or reasoning mode are unsupported.
• The askmdb CLI runs on Linux x86_64 (on Windows, through WSL 2). Its GPU support has been tested on NVIDIA only; it uses standard Vulkan, but AMD and Intel GPUs have not been tested yet. The GGUF itself runs anywhere llama.cpp does.

Changes from the base model

• A LoRA adapter trained for the contract above and merged into the base weights (question + schema block → one aggregate pipeline as JSON, with refuse and unsure).
• Tokenizer unchanged. Chat template: the Qwen3.5 template the model was trained with, which keeps reasoning off unless enable_thinking is true (the original Qwen3.5-9B template reasons by default).
• Language model only: no vision encoder and no multi-token-prediction head.
• Converted to GGUF and quantized to Q4_K_M with imatrix; general.* metadata added.

License

Apache-2.0, the same license as the base model Qwen3.5-9B. See LICENSE. The test set manuelgscom/askmdb-test is CC BY 4.0.

MongoDB is a trademark of MongoDB, Inc. askmdb is an independent project, not affiliated with or endorsed by MongoDB, Inc.