WaferSAGE-SFT: Qwen3-VL for Wafer Map Defect Understanding

This repository contains a supervised fine-tuned vision-language model for wafer map defect understanding.

It is part of WaferSAGE, a framework for wafer defect visual question answering via synthetic data generation, rubric-guided evaluation, and domain-specific post-training.

Historical note: this model was originally released under the WMVLM naming convention. It is now part of the WaferSAGE model family.

Model Summary

WaferSAGE-SFT is fine-tuned from Qwen3-VL on synthetic wafer map VQA data. The model is designed to answer natural language questions about wafer map images, including defect type identification, spatial distribution analysis, morphology description, and root-cause hypothesis generation.

Intended Capabilities

The model can answer questions such as:

• What type of defect pattern is visible on this wafer map?
• Where are the defective dies located?
• Is the defect concentrated near the center, edge, or a specific quadrant?
• Does the wafer show a scratch-like, ring-like, clustered, or random pattern?
• What process or equipment issue might be associated with this defect pattern?

Example Prompts

<image>
What type of defect pattern is visible on this wafer map?

<image>
Where are the defects located on the wafer?

<image>
Describe the morphology and spatial distribution of this wafer map defect.

<image>
What are the possible root-cause hypotheses for this defect pattern?

Training Data

This model was trained on WaferSAGE wafermap VQA data, generated through a multi-stage synthetic data pipeline:

Wafer map images + labels
    ↓
Clustering-based cleaning and sampling
    ↓
VLM-generated defect descriptions
    ↓
Structured rubric extraction
    ↓
VQA synthesis
    ↓
LoRA supervised fine-tuning

The VQA data covers:

• Defect type identification
• Spatial distribution
• Morphological description
• Root-cause hypothesis
• Consistency verification

Training Setup

The SFT models were trained with LoRA adaptation on Qwen3-VL.

Typical configuration:

Usage with Transformers

from transformers import pipeline

pipe = pipeline(
    "image-text-to-text",
    model="Niraya666/Qwen3-4B-wmvlm-260204"
)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "path_or_url_to_wafermap.png"},
            {"type": "text", "text": "What type of defect pattern is visible on this wafer map?"}
        ]
    }
]

output = pipe(text=messages, max_new_tokens=256)
print(output)

Usage with vLLM

vllm serve "Niraya666/Qwen3-4B-wmvlm-260204"

Then call the OpenAI-compatible endpoint:

curl -X POST "http://localhost:8000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  --data '{
    "model": "Niraya666/Qwen3-4B-wmvlm-260204",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Describe the wafer map defect pattern."
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "path_or_url_to_wafermap.png"
            }
          }
        ]
      }
    ],
    "max_tokens": 256
  }'

Evaluation

The model was evaluated using WaferSAGE's rubric-based wafermap VQA benchmark.

Evaluation dimensions include:

• Spatial understanding
• Morphological understanding
• Defect type recognition
• Root-cause hypothesis quality
• Hallucination avoidance

The evaluation combines:

• Rule-based rubric matching
• LLM-as-a-Judge scoring
• Qualitative error analysis

The SFT model primarily improves domain-specific terminology, response format, and wafermap-specific visual reasoning compared with the base VLM. For stronger performance, see the WaferSAGE RL models trained with GSPO and rubric-aligned rewards.

Limitations

This model is a domain-adapted VLM for wafer map understanding, but it has important limitations:

• It should not be used as a standalone root-cause diagnosis system.
• Root-cause outputs are hypotheses, not verified fab conclusions.
• It may hallucinate defect locations or process causes when the image is ambiguous.
• It may inherit biases from synthetic training data and teacher model outputs.
• It was trained primarily on public wafer map style data and may not generalize to all fab-specific wafer map formats.
• It does not use lot history, process metadata, tool/chamber records, metrology, or inline inspection data.

For production semiconductor engineering use, model outputs should be reviewed by qualified engineers and combined with process context.

Recommended Use

Recommended:

• Research on industrial VLMs
• Wafer map VQA experiments
• Defect pattern description
• Data generation and evaluation pipeline development
• Local proof-of-concept systems for semiconductor AI

Not recommended:

• Automated process control
• Final root-cause diagnosis
• Yield-impact decisions without expert review
• Safety-critical or high-cost manufacturing decisions without validation

Related Resources

• WaferSAGE paper: arXiv:2604.27629
• WaferSAGE VQA dataset: Niraya666/wafermap-vqa-2602
• Rubric-augmented datasets: Niraya666/wafermap-vqa-with-rubrics-2602, Niraya666/wafermap-vqa-with-rubrics-2602_v2
• RL model: Niraya666/Qwen3-VL-4B-Instruct-WMVLM-RL-0213

Citation

If you use this model, please cite:

@misc{xu2026wafersage,
  title        = {WaferSAGE: Large Language Model-Powered Wafer Defect Analysis via Synthetic Data Generation and Rubric-Guided Reinforcement Learning},
  author       = {Ke Xu and Zhongyuan Lian},
  year         = {2026},
  eprint       = {2604.27629},
  archivePrefix= {arXiv},
  primaryClass = {cs.AI}
}