Okinawa Sign VLM v2

An experimental vision-language model trained to read synthetic vocabulary cards containing Japanese prompts and attributed Shuri dictionary forms. The v2 training set contains 3,000 generated cards, five times the v1 example count.

Use

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

repo = "LoNebula/okinawa-sign-vlm-v2"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo)
image = ...  # PIL image or image URL loaded by your application
messages = [{"role": "user", "content": [
    {"type": "image", "image": image},
    {"type": "text", "text": "Read the Shuri word shown on this card."},
]}]
inputs = processor.apply_chat_template(messages, add_generation_prompt=True,
                                       tokenize=True, return_dict=True,
                                       return_tensors="pt")
generated = model.generate(**inputs, max_new_tokens=64)
generated = generated[:, inputs["input_ids"].shape[-1]:]
print(processor.batch_decode(generated, skip_special_tokens=True)[0])

Data and limitations

All images are original synthetic graphics generated by this project from source-attributed Shuri dictionary entries; they are not photographs or community-authored signs. The train/test split is grouped by Japanese headword so variants of one term do not cross splits. The 3,000 images represent the source vocabulary repeatedly in a controlled visual format, not 3,000 independent linguistic facts. This model has not been evaluated on real-world signs, handwriting, or varied layouts, and it may produce incorrect readings. The dataset card documents provenance and CC BY 4.0 attribution.

License

The fine-tuned weights are published under Apache-2.0, subject to the base model's terms. Dataset content and source-derived text have separate licensing; see the dataset card.