LingBot-Vision
LingBot-Vision is a family of self-supervised Vision Transformer backbones for dense spatial perception. The models are pretrained with masked boundary modeling, a boundary-centric objective that encourages spatially structured patch features while retaining strong semantic representations.
This Hugging Face repository stores a backbone-only PyTorch checkpoint as model.pt. It is intended for inference, feature extraction, PCA visualization, and downstream dense prediction research.
Model Details
Model Description
LingBot-Vision learns dense patch representations that preserve boundaries, shapes, and semantic regions. The backbone is trained from random initialization with self-supervised teacher-student pretraining. During training, teacher-discovered boundary tokens are forced into the masked set, and boundary tokens receive both semantic self-distillation and categorical boundary-field supervision.
The released model family includes:
- LingBot-Vision-Giant: ViT-g/16 backbone for highest-quality dense features.
- LingBot-Vision-Large: ViT-L/16 backbone for strong dense features and practical inference.
- LingBot-Vision-Base: ViT-B/16 backbone for balanced inference cost.
- LingBot-Vision-Small: ViT-S/16 backbone for lightweight demos and downstream use.
Each checkpoint contains backbone weights only. Training-time heads, optimizer states, and boundary-target generation components are not included.
- Developed by: Zelin Fu, Bin Tan, Changjiang Sun, Shaohui Liu, Kecheng Zheng, Yinghao Xu, Xing Zhu, Yujun Shen, Nan Xue
- Model type: Vision Transformer backbone for dense visual representation learning
- License: Apache 2.0
Model Sources
- Repository: https://github.com/robbyant/lingbot-vision
- Project Page: https://technology.robbyant.com/lingbot-vision
- Technical Report: coming soon
Related Models
- LingBot-Vision-Giant: https://huggingface.co/robbyant/lingbot-vision-vit-giant
- LingBot-Vision-Large: https://huggingface.co/robbyant/lingbot-vision-vit-large
- LingBot-Vision-Base: https://huggingface.co/robbyant/lingbot-vision-vit-base
- LingBot-Vision-Small: https://huggingface.co/robbyant/lingbot-vision-vit-small
Uses
Direct Use
- Dense Feature Visualization: Extract frozen patch tokens and visualize their PCA components.
- Image Feature Extraction: Use normalized patch tokens as spatial visual features.
- Backbone Initialization: Initialize downstream dense prediction models with LingBot-Vision weights.
Downstream Use
- Depth Estimation: Frozen patch tokens expose spatial structure to lightweight dense readouts.
- Semantic Segmentation: Boundary-faithful features help align region transitions with object contours.
- Video Object Segmentation: Frozen features support training-free label propagation and token matching.
- Depth Completion: LingBot-Vision can serve as the visual encoder initialization for LingBot-Depth 2.0.
How to Load
Install the LingBot-Vision inference repository and dependencies:
git clone https://github.com/robbyant/lingbot-vision.git
cd lingbot-vision
conda create -n lingbot-vision python=3.10 -y
conda activate lingbot-vision
python -m pip install -r requirements.txt
python -m pip install -e .
Load a pretrained backbone:
import torch
from lbot_vision_infer import load_pretrained_backbone
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32
backbone, embed_dim = load_pretrained_backbone(
variant="large",
device=device,
dtype=dtype,
)
print(backbone.patch_size, embed_dim)
The variant argument can be giant, large, base, or small. You can also pass an explicit Hugging Face model repo or a local directory to load_pretrained_backbone.
Technical Specifications
Model Architecture
- Backbone: Vision Transformer with patch size 16
- Released variants: ViT-g/16, ViT-L/16, ViT-B/16, ViT-S/16
- Output: Normalized patch tokens from the frozen backbone
- Checkpoint format: Backbone-only
.ptfile stored asmodel.pt - Training objective: Masked boundary modeling with self-distillation
Software Requirements
- Python >= 3.10
- PyTorch >= 2.0.0
- huggingface_hub
- omegaconf
Citation
@article{lingbot-vision2026,
title={Vision Pretraining for Dense Spatial Perception},
author={Fu, Zelin and Tan, Bin and Sun, Changjiang and Liu, Shaohui and Zheng, Kecheng and Xu, Yinghao and Zhu, Xing and Shen, Yujun and Xue, Nan},
year={2026}
}