Arxo Valentia ChemIQ 0.1 answers organic chemistry questions from a SMILES string without tools: it counts atoms and rings, finds the shortest path between two atoms, maps the atoms of two SMILES strings of one molecule, writes an IUPAC name, predicts a score from a structure-activity table and writes the product of a reaction. This is Arxo-Valentia-4B-ChemIQ-0.1, a LoRA adapter for Qwen/Qwen3.5-4B.
It follows the Arxo Materia releases on MedCalc-Bench and EHRSQL.
ChemIQ, all 816 questions, scored by the benchmark's own parser and answer checker. The model answers without tools.
| System | Trained on the task | ChemIQ | Source |
|---|---|---|---|
| Arxo Valentia ChemIQ 0.1 (4B) | yes | 605 / 816 (74.1%) | this model |
| GPT-5 | no | 70.2% | post by the benchmark's author |
| o3-mini, high reasoning | no | 56.7% | ChemIQ paper |
| Gemini 2.5 Pro | no | 55.5% | ChemIQ paper |
| DeepSeek-R1 | no | 50.2% | ChemIQ paper |
One training run. Greedy decoding, thinking switched off, a limit of 3,900 new tokens. Arxo Valentia was trained on questions of the same kinds, built from molecules outside the test set; the other rows are general models that answer from the prompt alone.
| Category | Questions | Arxo Valentia ChemIQ 0.1 (4B) | o3-mini, high | Gemini 2.5 Pro | DeepSeek-R1 |
|---|---|---|---|---|---|
| Carbon counting | 50 | 50 (100%) | 92.0% | 82.0% | 90.0% |
| Ring counting | 48 | 48 (100%) | 100% | 93.8% | 93.8% |
| Shortest path | 108 | 108 (100%) | 72.2% | 63.0% | 59.3% |
| Atom mapping | 184 | 144 (78.3%) | 56.0% | 46.2% | 54.3% |
| SMILES to IUPAC name | 200 | 141 (70.5%) | 28.5% | 38.5% | 20.0% |
| Structure-activity (Free-Wilson) | 40 | 34 (85.0%) | 100% | 95.0% | 95.0% |
| Reaction product | 90 | 80 (88.9%) | 60.0% | 55.6% | 45.6% |
| NMR structure elucidation | 96 | 0 (0%) | 38.5% | 51.0% | 38.5% |
| All | 816 | 605 (74.1%) | 56.7% | 55.5% | 50.2% |
The columns of the other models are computed from the model responses released with the benchmark, with the same checker; their totals match the paper.
Two notes on the numbers:
The adapter attaches to AutoModelForImageTextToText, not to AutoModelForCausalLM. The system prompt below is the one the model was trained with; the question follows the wording of ChemIQ.
import torch
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoTokenizer
base = "Qwen/Qwen3.5-4B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForImageTextToText.from_pretrained(base, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "arxohq/Arxo-Valentia-4B-ChemIQ-0.1")
SYSTEM = (
"You are a careful chemist. Work in the fixed step-by-step notation: write out the working first, "
"each step using only what is already written, then give the final answer alone on the last line "
"in exactly the form the question asks for."
)
question = (
"Write the IUPAC name of this molecule:\n\n"
"CC(=O)Nc1ccc(O)cc1\n\n"
"Write the IUPAC name only. Do not write any comments."
)
messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": question}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=3900, do_sample=False)
answer = tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(answer.strip().splitlines()[-1])
The model writes its working and then the answer alone on the last line. The benchmark's parser reads that last line.
Outputs for all 816 questions, per-question scores from the benchmark's checker and checksums: github.com/arxohq/arxo-valentia-chemiq
Qwen/Qwen3.5-4B