⚠️ Research / portfolio model. NOT for real pesticide advice. Its answers can be wrong in ways that harm crops, people and livestock. Always follow the product label, which is legally binding in India, and consult a local agriculture officer or a qualified agronomist.
The model takes a farmer's question about a pest or disease and returns one structured JSON advisory. The advisory gives likely causes, non-chemical measures first, then chemical options with dose, basis, pre-harvest interval (PHI) and cautions. It also says whether to escalate to an expert.
Adapter. LoRA on the 4-bit base model, trained with Unsloth + TRL SFT (per the adapter's saved metadata).
| setting | value |
|---|---|
| base model | unsloth/Qwen2.5-7B-Instruct-bnb-4bit |
| LoRA r / alpha / dropout | 32 / 64 / 0 |
| target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| epochs | 1 (from session record) |
| steps | 281 (from session record) |
| learning rate | 2e-4, cosine schedule, 20 warmup steps (from session record) |
| batch size | 2 per device × 8 gradient accumulation = 16 effective (from session record) |
| optimizer / precision / seed | adamw_8bit, weight decay 0.01 / fp16 / 42 (from session record) |
| max sequence length | 2048 (from session record) |
| GPU | Kaggle T4 (from session record) |
| training / held-out test examples | 4,493 / 1,136 |
The training cell ran on Kaggle and was never committed. The session-record
values have not been verified against a saved trainer_state.json; see
reports/training_config.md in the GitHub repository.
The training data is synthetic:
claude-sonnet-4-6) from those
queries plus the matching CIB&RC label rows.All scores come from the project's own verifier, on a 498-item benchmark (500 items minus 2 with known defects). Each score is between 0 and 1.
The arms:
| arm | what it is |
|---|---|
| R0 | the base model, no fine-tuning |
| R3 | this adapter |
| R2 | the base model, shown the output schema plus a fact sheet of CIB&RC label rows retrieved for the query |
| R2 unwrapped | R2 re-scored with its {"Advisory": …} wrapper removed — a scoring check, not a model run |
| R4 | this adapter, plus the same fact sheet |
"cap" is max_new_tokens; "excl" counts items excluded on documented label
contradictions.
| arm | cap | mean | scored / excl / missing | G1 % | G2 % | C1_dose | C2_phi | C3_formulation | C4_escalation | C5_causes_top1 | C6_non_chemical | hit_limit |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| R0_baseline | 1,200 | 0.1726 | 498 / 0 / 0 | 100.0 | 53.0 | 0.0000 | 0.0000 | 0.0000 | 0.8182 | n/a | 0.0000 | 0 |
| R3_sft | 1,200 | 0.5220 | 487 / 11 / 0 | 99.6 | 99.4 | 0.3675 | 0.3134 | 0.4636 | 0.7913 | 0.6746 | 0.5385 | 2 |
| R2_base_rag | 3,072 | 0.2060 | 497 / 1 / 0 | 100.0 | 66.7 | 0.4575 | 0.4592 | 0.1219 | 0.4139 | 0.5556 | 0.7732 | 0 |
| R2_base_rag_unwrapped | 3,072 | 0.2912 | 487 / 11 / 0 | 100.0 | 80.1 | 0.6370 | 0.6459 | 0.1583 | 0.4665 | 0.5769 | 0.7686 | 0 |
| R4_sft_rag | 3,072 | 0.5840 | 498 / 0 / 0 | 100.0 | 91.0 | 0.3926 | 0.4204 | 0.4215 | 0.7991 | 0.6236 | 0.4122 | 0 |
Paired comparisons, on items scored in both arms:
| A vs B | n | mean A | mean B | Δ (A−B) | 95% CI | wins / losses / ties | sign test p |
|---|---|---|---|---|---|---|---|
| R4_sft_rag vs R2_base_rag | 497 | 0.5852 | 0.2060 | +0.3792 | [+0.3236, +0.4337] | 332 / 76 / 89 | 3.07e-39 |
| R4_sft_rag vs R2_base_rag_unwrapped | 487 | 0.5791 | 0.2912 | +0.2879 | [+0.2306, +0.3446] | 307 / 104 / 76 | 2.38e-24 |
| R4_sft_rag vs R3_sft | 487 | 0.5807 | 0.5220 | +0.0586 | [+0.0229, +0.0939] | 105 / 59 / 323 | 4.08e-04 |
| R2_base_rag_unwrapped vs R0_baseline | 487 | 0.2912 | 0.1730 | +0.1183 | [+0.0710, +0.1667] | 172 / 134 / 181 | 0.034 |
The GitHub repository includes a post-generation guard (src/guard.py). It:
Counts are options / items.
| R4 | before guard | after guard |
|---|---|---|
| mean (498 items) | 0.5840 | 0.6597 |
| G2 (schema-valid) % | 91.0 | 100.0 |
| overdoses (WRONG_HIGH, RANGE_TOP) | 12 / 10 | 1 / 1 |
| per-hectare dose labelled per acre | 10 / 5 | 0 / 0 |
| unregistered chemicals (G5) | 3 / 1 | 1 / 1 |
| unknown-PHI options not escalated | 87 / 45 | 0 / 0 |
| false positives (removed options the verifier grades correct) | — | 0 (of 2 unregistered + 32 dose removals) |
The one overdose left after guarding (item B1_17) is a verifier rounding defect; the dose itself matches the label.
The guard reads the same label data as the scorer, so its score gain shows consistency between the two, not independent proof that the answers are right.
{"Advisory": …}. This adapter
wrapped none.reports/phase14_clean_eval.md §10 in the GitHub
repository.The R4 numbers above used a CIB&RC fact sheet in the user message, built by
src/retrieval.py in the GitHub repository. A bare query, as below,
corresponds to R3.
tools/generate.py in the repository is the exact script used for
evaluation. It uses Unsloth.
from huggingface_hub import hf_hub_download
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
BASE = "unsloth/Qwen2.5-7B-Instruct-bnb-4bit" # 4-bit: needs bitsandbytes and a CUDA GPU
ADAPTER = "Janhvi1008/agri-llm-qwen2.5-7b-lora"
tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)
# The frozen system prompt the adapter was trained with.
system = open(hf_hub_download(ADAPTER, "system_prompt.txt"), encoding="utf-8").read()
messages = [{"role": "system", "content": system},
{"role": "user", "content": "Whitefly attack on my cotton, what should I spray?"}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**inputs, max_new_tokens=3072, do_sample=False, num_beams=1,
repetition_penalty=1.05) # greedy, as evaluated
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
unsloth/Qwen2.5-7B-Instruct-bnb-4bit