Spark-X2.6-4B-SQL-Cypher-Agent (GGUF)

GGUF quants of trjxter/Spark-X2.6-4B-SQL-Cypher-Agent, a 4B tool-calling agent that answers questions about a database by writing one read-only query: SQLite SQL against a relational schema or Neo4j Cypher against a graph schema. It thinks first, then calls whatever query tool you give it. It's a LoRA fine-tune of XHToken/Spark-X2.6-4B, merged into the base weights, trained on a single 16 GB consumer GPU (AMD RX 7800 XT).

Needs llama.cpp b10828 or newer (the Spark architecture, spark2_5). The chat template is embedded, and thinking is on by default.

Files

Also here: Spark-X2.6-4B-SQL-Cypher-Agent-imatrix.gguf (7 MB), the importance matrix the K-quants were made with. Pass it to llama-quantize --imatrix to make other sizes with the same calibration.

Which one? Q8_0 if it fits, since it's practically lossless. Q6_K_XL is the best of the smaller ones. Q4_K_XL is the pick when memory is tight: half the error of Q4_K_M for 250 MB more.

• KLD and "same top token" measure how closely a quant matches the BF16 model's next-token distribution: mean KL divergence, and how often both pick the same top token. They're measured on held-out validation text in the model's own format, 24 chunks of 2,048 tokens, none of it used for the importance matrix. This is the better guide to fidelity.
• Evalset accuracy is execution accuracy on a separate 200-question evalset, with greedy decoding and a 1,024-token thinking budget. The unmerged adapter on the bf16 base scored 45.4% there. With 200 questions, about ±3.5 points is noise, so these five numbers don't rank the quants.

Results

Execution accuracy on a held-out test set of 1,000 questions, 250 from each dataset's own test split. None of the questions appear in training. The same harness ran both models: llama.cpp, the same prompts, greedy decoding, a thinking budget of 1,024 tokens.

+10.7 points overall (95% interval ±2.9). Compared question by question on the 856 questions both runs could score: the fine-tune alone got 129 right, the base alone got 37 right (sign test p ≈ 4e-13). The unmerged adapter on the bf16 base scored 43.9% on the same test, so merging and quantising to Q8_0 cost nothing measurable.

How an answer is scored. A question counts only if the query returns the same result as the dataset's reference query, not if it merely looks similar:

• gretel: both queries run on the example's real rows.
• SQaLe: both run on several randomly filled copies of the schema. This is the test-suite idea from Spider: two queries that agree on several random databases very likely mean the same thing. Some copies place values right next to the query's literals, so > against >= or LIKE against = give different rows.
• CypherBench: both run on the dataset's own sampled graphs in Neo4j.
• text2cypher: both run on random graphs built from the prompt's schema.

Results are compared as multisets of rows, and row order counts when the reference sorts. A question where no database tells the two queries apart is left out, which is why the scored counts are just under 1,000. Exact-match scores would be far lower and mean little here.

Running it

llama-server -m Spark-X2.6-4B-SQL-Cypher-Agent-Q8_0.gguf -ngl 99 -fa on -c 8192 --reasoning-budget 1024

The model expects the setup it was trained on:

• a system message with the schema, optional example queries and a few rules;
• the question as the user message;
• one query tool in tools.

The tool's name and argument don't matter (training used run_query(query), execute_sql(sql), cypher_query(cypher), ...), and unrelated tools in the list are ignored. If the schema can't answer the question, the model is trained to say so in one sentence instead of calling the tool.

A request to llama-server's OpenAI-compatible /v1/chat/completions:

import requests

schema = """CREATE TABLE teams (id INTEGER PRIMARY KEY, name TEXT, city TEXT);
CREATE TABLE games (id INTEGER PRIMARY KEY, season INTEGER, home_id INTEGER, away_id INTEGER,
                    home_score INTEGER, away_score INTEGER);"""
system = f"""You answer questions about the database below by calling the `run_query` tool with ONE read-only SQLite SQL query.

{schema}

Examples (tested queries on this database):
(none)

Rules:
- One statement, read only (SELECT or WITH ... SELECT). No comments. The query runs as written, so use literal values, not parameters.
- Return named columns: the names and the numbers the answer needs. Never return whole rows of every column unless the question asks for them.
- Match names and values as the data spells them.
- Return exactly the rows the question asks for, in its shape: one row for a single answer ("which is the most", "the highest": ORDER BY the measure, LIMIT 1); one row per entity for "which ...", "every", "list"; the top N for "top N".
- If the database can't answer the question, don't call the tool: say why in one short sentence."""
tools = [{"type": "function", "function": {
    "name": "run_query",
    "description": "Run one read-only SQLite SQL query against the database and return the result rows.",
    "parameters": {"type": "object", "properties": {"query": {"type": "string", "description": "The query text."}},
                   "required": ["query"]}}}]

r = requests.post("http://localhost:8080/v1/chat/completions", json={
    "messages": [{"role": "system", "content": system},
                 {"role": "user", "content": "Which team scored the most points at home in 2024?"}],
    "tools": tools,
    "temperature": 0,
}).json()
msg = r["choices"][0]["message"]
print(msg.get("reasoning_content"))                         # the thought
print(msg["tool_calls"][0]["function"]["arguments"])       # {"query": "SELECT ..."}

For a graph, use the same shape with "graph database", "Neo4j Cypher", MATCH ... RETURN and "nodes", and give the schema as node labels, relationship types and their properties.

• Thinking is on by default. Thinking options go in "chat_template_kwargs": {"enable_thinking": false} turns thinking off, {"reasoning_effort": "low"} shortens it. Spark's template accepts only high and low, so don't set the server-wide --reasoning-effort medium. The scores above use thinking with a 1,024-token budget; the median thought is about 650 characters.
• Decoding. Every score above uses greedy decoding (temperature: 0). The embedded sampling default (temperature 1.0, top-p 0.95) suits chat better than query writing.
• The raw text the model writes looks like ...</think><tool_call>run_query<arg_key>query</arg_key><arg_value>SELECT ...</arg_value></tool_call>; llama-server parses it into reasoning_content and tool_calls.

Training

• Method: LoRA, rank 16 / alpha 32 / dropout 0.05, on every linear layer of every block (q_k_v_proj, out_proj, gate_proj, up_proj, down_proj). AdamW, lr 1e-4, 3% warmup, cosine decay, weight decay 0.01, batch 32, max length 4,096, seed 42. The loss is on the assistant turn only.
• Data: 94,745 examples, about 66M tokens, one epoch.
  • 90,549 gold calls: each dataset's reference query, written as a thinking-off tool call. SQaLe 27,887, gretel 28,695, text2cypher 25,871, CypherBench 8,096.
  • 4,196 self-generated traces: the base model's own thought and tool call on training questions, kept only when the query passed the execution checks above (thinking high 2,829 / low 1,129 / off 238). These teach it to reason before calling without drifting from its own style.
• Checkpoint chosen: optimizer step 1,500 of 2,961. A snapshot was scored every 500 steps on the 200-question evalset. The final adapter and the best mid-run one (step 1,500) were both run on the test set. Step 1,500 scored 43.9% against 42.2% for the final one, and both held-out sets pointed the same way, so step 1,500 shipped.
• Hardware: one AMD Radeon RX 7800 XT (16 GB, ROCm) under WSL, about 24 hours for the run. The pipeline (trace generation, checking, training, evaluation) is a personal project.

How the quants were made

• Converted from the merged bf16 model with llama.cpp's convert_hf_to_gguf.py (b11779), then quantised with llama-quantize from the same build.

• Importance matrix for the K-quants, from about 242k tokens of the model's training examples rendered in its chat format: thoughts, tool calls, schemas, from all four datasets. Q8_0 doesn't use one.

• XL is this repo's own recipe, not Unsloth's UD. It's the K-quant with two upgrades:

  • the tied token embedding, which doubles as the output head (8.2% of the weights), at Q8_0;
  • every attention matrix (attn_qkv, attn_output) one step up: Q8_0 in Q6_K_XL, Q6_K in Q4_K_XL.

  The feed-forward bulk (69% of the weights) stays at the base type. For Q4 it halves the KLD. The commands, for reference:

  llama-quantize --imatrix imatrix.gguf --token-embedding-type q8_0 \
    --tensor-type attn_qkv=q8_0 --tensor-type attn_output=q8_0 model-BF16.gguf model-Q6_K_XL.gguf Q6_K
  llama-quantize --imatrix imatrix.gguf --token-embedding-type q8_0 \
    --tensor-type attn_qkv=q6_k --tensor-type attn_output=q6_k model-BF16.gguf model-Q4_K_XL.gguf Q4_K_M
  

Limitations

• About 44% of answers return the reference result. It's a 4B model. Run its queries read-only, and check them before trusting a number, especially on unfamiliar schemas.
• One query per answer, one turn. It wasn't trained to read a tool result and try again (multi-step agent loops are the next stage of the project).
• Dialects: trained on SQLite SQL and Neo4j Cypher. Other SQL dialects work as far as their syntax overlaps with SQLite.
• English only, and questions in the style of the four datasets: single-database analytics, joins, groups, top-N, filters.
• It sometimes adds SELECT DISTINCT where the reference doesn't, which changes row counts on duplicate-heavy tables.
• text2cypher is the weakest set (27.7%); its databases aren't public, so its answers are checked on random graphs, a harder and noisier test.

Licence and attribution

Apache-2.0, like the base model. Base model: XHToken/Spark-X2.6-4B (Apache-2.0).

Training data, used under their licences: