Locus-2B

Locus-2B grounds a referring expression in an image and returns its bounding box, or answers that the object is not in the image. It is Qwen/Qwen3-VL-2B-Instruct with a small box head, trained with the Locus (Locate Once, Check Until Settled) recipe:

1. The image is prefilled once; every expression about it runs as a short branch over the cached prefix.
2. The box head drafts the box in one pass.
3. The draft is rendered as the coordinate tokens the model would write, and the model verifies all of them in parallel by Jacobi iteration until the draft stops changing.

With exact acceptance the returned box is the model's own greedy coordinate-token answer, reached in a few passes instead of one decoding step per token.

This is version v1.1 (November 2026).

Versions

main is always the latest version, and each version is a tag. To load an earlier one:

m = from_pretrained("<org>/Locus-2B", revision="v1.0")   # or: hf download <org>/Locus-2B --revision v1.0

The current code loads every version; v1.0 code cannot read v1.1 weights. To keep v1.1 but always get a box, as in v1.0, pass allow_none=False to ground. Full list of changes: the code repository's CHANGELOG.md.

Files

This repository holds only the weights training changed: the box head and the fully fine-tuned language model and vision merger (model.safetensors), plus config.json (architecture, version, defaults). The vision encoder, tokenizer and remaining weights come from Qwen/Qwen3-VL-2B-Instruct, which from_pretrained downloads separately.

Usage

from PIL import Image
from locus.hub import from_pretrained, ground   # pip install -e <code repo>

m = from_pretrained("<org>/Locus-2B", graphs=True)
for r in ground(m, Image.open("street.jpg"), ["the man in the red shirt", "the bus on the left"]):
    print(r["expression"], r["box"], r["no_target"])   # box = [x0, y0, x1, y1] in pixels

accept=1.0 (default) returns the greedy coordinate-token box; accept=0.5 uses typical acceptance (fewer passes, no longer exactly greedy); verify=False returns the head's one-pass draft. graphs=True replays CUDA graphs for shapes seen before. A no-target answer sets no_target=True and box=None; ground(..., allow_none=False) (or allow_none: false in config.json) returns the head's draft as the box instead, as v1.0 always gave a box. A CUDA GPU is required.

Results

Mean over the 8 RefCOCO / RefCOCO+ / RefCOCOg val and test splits, at 640² pixels. Passes: mean sequential passes per expression.

For reference, on the same rows: Jag-0.8B (regression head) 88.16 / 76.51 / 48.76; LocateAnything-3B (box tokens) 88.68 / 80.82 / 65.55.

Through ground:

Latency, one RTX A6000, batch 1, RefCOCO testA at 640², mean ms per request: 226 without graphs, 111 with graphs=True on a first-seen image shape, 98 once shapes repeat.

Training

Qwen3-VL-2B-Instruct with full fine-tuning of the language model and vision merger at lr 1e-5, head and verifier trained jointly, 48k steps of 4 images x 4 expressions. Training data: the RefCOCO, RefCOCO+ and RefCOCOg train splits, plus 15% added rows: gRefCOCO no-target expressions, and category-contrast, small-target and noun-swap rows built from COCO-2017 instance annotations. Items are augmented with pasted COCO occluders (p = 0.3), zoom-out (0.15), request rephrasing (0.1) and photometric changes (0.15). Every image that appears in any evaluation split is removed from all of it.

Limitations

• RefCOCO accuracy is slightly below v1.0.
• Small objects are still the weak case (table above); a second pass at 1024² helps.
• "Not in the image" is answered for part of the no-target expressions only; the rest get a box.
• Verification is slower than the head alone, and slower than a pure regression head.
• Exactness holds for this model's greedy decode; in bf16 it holds up to the rounding that also makes two greedy runs with different batch shapes differ.
• The model answers grounding queries only. Other questions should go to the base model.

License and data

Weights: Apache-2.0, derived from Qwen/Qwen3-VL-2B-Instruct (Apache-2.0). Training annotations are RefCOCO/+/g and gRefCOCO over COCO images; check those datasets' terms for your use.