Okinawa Sign & Shuri Vocabulary Reader

This compact vision-language prototype reads Japanese vocabulary displayed on synthetically rendered Okinawa-style learning signs and returns the corresponding Shuri Okinawan dictionary form. It is a small, reproducible experiment for dialect learning.

The training images are generated graphics, not photographs of real Okinawan signs. This model has not been evaluated on street photography, place-name interpretation, or real-world OCR. It is not a general-purpose sign reader.

Quick start

pip install torch transformers pillow

from PIL import Image
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

repo = "LoNebula/okinawa-sign-vlm"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo)
image = Image.open("sign.png").convert("RGB")

messages = [{
    "role": "user",
    "content": [
        {"type": "image"},
        {"type": "text", "text": (
            "Read the Japanese vocabulary shown in this Okinawa-style learning sign. "
            "Give its Shuri Okinawan dictionary form. Reply with only the dictionary form."
        )},
    ],
}]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(text=prompt, images=[image], return_tensors="pt")
device = next(model.parameters()).device
inputs = {key: value.to(device) if hasattr(value, "to") else value for key, value in inputs.items()}
with torch.inference_mode():
    generated = model.generate(**inputs, max_new_tokens=48, do_sample=False)
prompt_length = inputs["input_ids"].shape[1]
print(processor.batch_decode(generated[:, prompt_length:], skip_special_tokens=True)[0])

Training data and provenance

The image files are original graphics generated locally by this project. The Japanese prompts and Shuri dictionary forms come from LoNebula/okinawa-ja-shuri-dictionary-translation, a Shuri-only, attributed derivative of the National Institute for Japanese Language and Linguistics' Okinawa-go Jiten index. The source dataset is marked CC BY 4.0. Each training example preserves its source ID, citation, source URL, and license. See the dataset card for the full attribution and generation recipe.

Base model and training

• Base: HuggingFaceTB/SmolVLM-500M-Instruct
• Fine-tuning: supervised instruction tuning with a LoRA adapter, merged for direct Transformers loading.
• Dataset: LoNebula/okinawa-sign-vlm-dataset; 600 generated image-text examples with a fixed 90/10 train/test split.
• The model repository follows the base model's Apache-2.0 license. The separately published training dataset is CC BY 4.0; retain its source attribution when reusing or redistributing that dataset.

Intended use and limitations

• Use for experimentation with synthetic Shuri vocabulary cards and dialect-learning interfaces.
• Outputs use the source dictionary's romanization and symbols; they are not standardized kana spellings or natural conversation.
• Dictionary entries can be ambiguous or context-dependent. The model may return an unsuitable form.
• The small synthetic dataset does not establish real-world OCR or translation quality. Do not rely on the model for navigation, safety signs, or formal linguistic work without verification.
• The fixed split can contain a repeated Japanese headword across train and test. Held-out loss is exploratory and is not an estimate of generalization to unseen vocabulary.
• No speech recognition, audio generation, or real sign-photo data is included.

Attribution

National Institute for Japanese Language and Linguistics, Okinawa-go Jiten (1963; index), as packaged by doraking/ryukyuan-okinawan-corpus, CC BY 4.0. Source: https://mmsrv.ninjal.ac.jp/okinawago/. Images were generated by the dataset preparation script in this project.