Molmo2-O-7B, think symbolic image

Molmo2-O-7B, think symbolic image is a LoRA adapter of allenai/Molmo2-O-7B, Ai2's 7.8-billion-parameter vision model (the Molmo vision encoder and the OLMo 3 7B language model of 32 layers, trained on dense human picture captions), that looks at a picture and writes the row of thought-symbolic-image-captions: a short chain of plain-language thoughts, one named step per line, then the wordings (a caption, a prompt, tags, a critique, by depth), then the questions a careful looker would still ask, each with its answer and what settles it. It is one of five model families trained beside the Lucia line (Qwen3.5) and the Ulisse line (Gemma 4) under Lucia v1's recipe and read on the same held-out sets; it is not named yet. Every looker is taught the same rows under the same recipe; the base models differ in what they can hold, and the reads below say what this one holds. Its base model answered the row's form only a third of the time; the adapter writes it whole and backs its thoughts at Lucia's level, and the medium term reads at Ulisse's level.

see: A single figure, seated, holding a book, in three-quarter view.
place: The figure sits at the centre, the book held low against the lap.
form: The sitter fills the frame from the knees up, the background a plain dark wash.
colour: Brown ink over graphite, the paper a warm cream.
mood: The lowered eyes and the still hands suggest absorption, a reader lost in the page.
caption: A seated figure reads a book held in the lap, drawn in brown ink over graphite.
prompt: drawing, seated figure reading, three-quarter view, brown ink over graphite, cream paper, plain dark wash
question: thing name -> book [record]
question: medium -> ink and graphite [record]
question: mood tone -> absorbed [chain]
question: sitter identity -> unknown [unknown]

The steps a chain may use, in the order a careful looker takes them: see, name, place, count, relate, form, light, colour, mood, mean, intend, story, compare, change, check, say. A quick look holds 1 to 2 thoughts, a closer look 3 to 5, a long look 6 to 10. Questions end in [record] (the answer is a fact of the picture), [chain] (the chain worked it out) or [unknown]. The prompt line's first term names the kind of picture (painting, print, drawing, photograph).

Loading

This repository holds the LoRA adapter (rank 32, 67.4 M trainable parameters, saved in fp32); there is no full fine-tune. Its model code is the base repository's own (trust_remote_code), pinned here to the revision it was trained on and written for transformers 4.57: install transformers==4.57.6 and peft 0.18 or newer (the training and the reads ran on exactly these).

Run it as it was trained: greedy decoding, no system prompt, no thinking (the base is not a reasoning model). Its rows end with <|endoftext|>, the id its generation config stops on. The processor comes from the base repository at the same revision; nothing of the base's code is copied here.

The base model, the adapter merged into it, the processor from the base repository at revision 78441065, the revision the adapter was trained on:

import torch
from huggingface_hub import snapshot_download
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

src = snapshot_download("allenai/Molmo2-O-7B", revision="784410650d12be9bc086118fdefa32d2c3bced86")
processor = AutoProcessor.from_pretrained(src, trust_remote_code=True)
processor.tokenizer.padding_side = "left"   # its processor places its start token assuming left padding
model = AutoModelForImageTextToText.from_pretrained(src, trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda")
model = PeftModel.from_pretrained(model, "AbstractPhil/molmo2-o-7b-think-symbolic-image").merge_and_unload().eval()

Then:

from PIL import Image

image = Image.open("picture.jpg").convert("RGB")
prompt = ("Look at this picture and think it through in writing, a closer look; then write a caption and a "
          "prompt or tags, and the questions a careful looker would still ask, each as `question: what -> "
          "answer [record, chain or unknown]`.")
messages = [{"role": "user", "content": [{"type": "image", "image": image}, {"type": "text", "text": prompt}]}]
text = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=800, do_sample=False)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])

The prompts it was trained on

The adapter was trained on these exact asks (the trained forms of the package's prompts.full and prompts.short); a reworded ask is a different input. The reads below use the whole row at each picture's depth.

The depth words: "a quick look" (1 to 2 thoughts), "a closer look" (3 to 5), "a long look" (6 to 10). The parser, the scorer and the trainer are in qwen-symbolic-thinking (qwen_symbolic_thinking.chain.parse, chain.score); the same package extracted every number below.

The reads

Read the same way as the Lucia and Ulisse lines, by the same reader, beside Lucia v1 (the model the recipe was made for) and Ulisse 12B. First a screen: 60 pictures and 40 specifications no training row came from, scored against the dataset's own rows. Row shape is the share of rows written in the row's form; at its depth, the share whose chain holds the number of thoughts its depth asks for; said is the share of a picture's own captions' content words that the thoughts and the caption say, and backed the share of the model's content words those captions back; a definition is a thought that defines a word instead of looking. Then the medium term on two sets no training row came from, from the picture list AbstractPhil/qwen-thinking-images: 500 National Gallery of Art pictures (100 drawings, prints, paintings, photographs and objects each), the prompt line's first term read against the catalogue's kind, and 200 pictures of the Art Institute of Chicago, the National Gallery, the Cleveland Museum of Art and the Met.

What this says: A broken base form became Lucia's: the adapter writes the row's shape every time and at its depth on every picture row (2.6 thoughts a closer look; the specification rows 0.90, the lowest of the five). The grounding comes up to her band: it backs 0.28 of what it says (its base 0.15; Lucia v1 0.30) and says 0.18 of a picture's own captions (hers 0.19). The medium term sits at Ulisse's level: 408 of the 500 held-out pictures (Ulisse 12B 409, Lucia v1 415), every kind within 4 of hers, prints the furthest (73 against 76); the museum pictures within 1 a kind of hers. Its specification rows define the most (16 of 40; Lucia 3), and its picture rows reach for the stock mood words most (81% of its mood lines; Lucia 46%). It reads in batches of 32 (it keeps about 512 KB of cache a token), at 0.63 seconds a picture, and took 338 minutes to train.

Before training the base model answered the row's form 0.35 of the time from the plain ask, wrote 11.8 thoughts a picture (0.17 of rows at their depth), and backed 0.15 of what it said. The training taught it the row's shape at each depth, the backing and the house vocabulary.

Limits, shared with the family: quick looks are the fragile depth, where one thought can name a wrong thing; the museum painting row understates every model, because watercolours and gouaches on paper are named drawings, as the National Gallery's own catalogue does; English only; the row's conventions (the steps, the depths, the question form) are the dataset's. The specification-row definitions are counted on 40 rows, the noisiest line of the table. The reads are made with qwen_symbolic_thinking.tools (probe_read, museum_read, museum_mismatch, screen_conflicts), so any adapter can be read the same way.

How it was trained

Where it was trained

• The code: qwen-symbolic-thinking on GitHub: the trainer, the screens, the reads and what each model family needs (qwen_symbolic_thinking/family.py). This adapter was trained by v0.1.37 (tag v0.1.37); the batching and loading notes on this card hold for v0.1.38.
• The run's record: the model repository AbstractPhil/qwen-symbolic-thinking-image-training (a model repository, not a dataset): this adapter with its step checkpoints under lora/Molmo2-O-7B/r32/20261102_123335_G_med4_2ep_molmo2o_v4/, the training sets it read (sft/pictures_v2.json, sft/drawn_deep_v2.json, synth/S_synth.json, sft/medium_v4.json) and the screens under screens/.
• The rows: the dataset thought-symbolic-image-captions (the row's form and the picture rows) and the picture list AbstractPhil/qwen-thinking-images (the museum pictures and the held-out probe).

Versions

The lookers

The Lucia line is Qwen3.5 and starts at AbstractPhil/qwen3.5-9b-think-symbolic-image; the Ulisse line is Gemma 4. The five families below them (North-Micro-Vision, GLM-4.6V-Flash, Molmo2-O-7B, Ministral 3 14B and Phi-4-reasoning-vision-15B) are trained under Lucia v1's recipe at her learning rate (each has its own vision encoder, frozen under the adapter) and read on the same sets.

License

Apache-2.0, the base model's licence. The rows it was taught come from the datasets above under their own terms.