ComfyUI-WeVisOne-VL

ComfyUI nodes for tencent/WeVisOne-VL-4B: open-vocabulary detection, referring localization, pointing, counting, OCR, and promptable alpha matting.

The model is a Qwen3-VL-4B with 1010 extra tokens and its own lm_head, plus a 9.6M-parameter pixel decoder. The pack reuses ComfyUI's native Qwen3-VL implementation, and the decoder runs on ComfyUI ops.

Examples


detection: `dog`	detection: `sofa` `pillow` `vase` `painting`	detection: `guitar` `drum` `microphone`
referring: `the boy wearing glasses`	count_point: `white chess pieces` (15)	ocr
matting: `person`	matting: `the red parasol`	matting: `the boy playing the drums`

Input images are the demo images of Tencent/WeVisOne-VL. The overlays are drawn from the node's raw output; mattes are shown by dimming the background.

Install

Clone the pack without the weights, then download the weights into models/text_encoders:

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/paralaif/ComfyUI-WeVisOne-VL custom_nodes/ComfyUI-WeVisOne-VL
hf download paralaif/ComfyUI-WeVisOne-VL wevisone_vl_4b_int8_convrot.safetensors --local-dir models/text_encoders

file	size
`wevisone_vl_4b_int8_convrot.safetensors`	5.7 GB	int8 + ConvRot language and vision linears (comfy-kitchen format), about 3× faster than bf16 on a 16 GB GPU; boxes within 2/1000 and mattes at 0.999 IoU of bf16
`wevisone_vl_4b_bf16.safetensors`	9.7 GB	the original bf16 weights

Both files bundle the fp32 pixel decoder. To build the bf16 file yourself from the original release:

hf download tencent/WeVisOne-VL-4B --local-dir WeVisOne-VL-4B
python custom_nodes/ComfyUI-WeVisOne-VL/convert_wevisone.py WeVisOne-VL-4B models/text_encoders/wevisone_vl_4b_bf16.safetensors

Nodes

WeVisOne-VL Loader loads the packed file from models/text_encoders and outputs a CLIP. The core Generate Text node also works with it.
WeVisOne-VL Perceive takes an image, a task and a query. It outputs:
- the raw response;
- bboxes, compatible with SAM3 Detect and Draw BBoxes;
- points as JSON, usable as SAM3 point prompts;
- mask: the alpha mattes for the matting task, or filled boxes for the other box tasks. It gives their union per image, or one mask per instance with individual_masks.

task	query
detection, point	labels, one per line
referring, matting	a referring phrase
count_box, count_point	the object to count
ocr	—
ocr_grounding	the text to find
freeform	the full prompt

Decoding is greedy with the reference repetition penalty of 1.05. The pixel decoder runs in fp32 on a 1024×1024 canvas and needs about 2 GB of VRAM per instance.

example_workflows/WeVisOne-VL-Perceive.json mattes a person and builds a transparent cutout; copy example_workflows/wevisone_seabreeze.jpg into input/ to run it as is.

License

Apache 2.0. The weights are converted from tencent/WeVisOne-VL-4B (Apache 2.0), and the pixel decoder is ported from the reference implementation.

@article{liang2026wevisone,
  title={WeVisOne-VL: Unified Visual Perception from Objects to Pixels},
  author={Liang, Tianming and Pan, Junyi and Li, Yuxi and Fu, Canmiao and Huang, Zhipeng and Sun, Chong and Li, Chen and Lyu, Jing},
  journal={arXiv preprint arXiv:2610.31329},
  year={2026}
}