LingBot-Vision: Vision Pretraining for Dense Spatial Perception
🔭 Meet LingBot-Vision! A family of self-supervised ViT backbones for dense spatial perception! 🖼️📐
LingBot-Vision is a family of self-supervised Vision Transformer backbones for dense spatial perception, from ViT-S/16 up to a 1.1B-parameter ViT-g/16. The flagship model is pretrained with masked boundary modeling — a boundary-centric objective that encourages spatially structured patch features while retaining strong semantic representations.
Boundary-centric masked modeling. Each row shows the input image, the PCA projection of frozen patch tokens, teacher-discovered boundary tokens, and cosine-similarity maps from selected boundary-token queries. The features capture semantic grouping and geometric structure at the same time.
LingBot-Vision learns boundaries, shapes, and semantic regions all together, making it a drop-in visual encoder for dense downstream tasks:
- 🎨 Dense feature visualization — PCA maps of frozen patch tokens reveal coherent object regions and crisp boundaries
- 📏 Depth estimation — frozen patch tokens expose spatial structure to lightweight dense readouts
- 🧩 Semantic segmentation — boundary-faithful features align region transitions with object contours
- 🎬 Video object segmentation — training-free token matching and label propagation with frozen features
- 🤖 Depth completion — LingBot-Vision is the visual encoder initialization for LingBot-Depth 2.0 (see below)
🌊 Meet LingBot-Depth 2.0
By simply replacing the encoder with LingBot-Vision at the ViT-L/16 and ViT-g/16 scales, and scaling the curated RGB-D training corpus from 3M to 150M samples, LingBot-Depth 2.0 achieves substantial performance gains over the previous and other system, as detailed in the technical report.
LingBot-Depth 2.0 on mirror and glass scenes. Each group shows input RGB, raw sensor depth, refined depth, and refined point clouds. Raw depth is missing on difficult surfaces such as window panes, glass balustrades, and reflective floors.