LingBot-Vision is a family of self-supervised Vision Transformer backbones pretrained with masked boundary modeling for dense spatial perception tasks like depth estimation and segmentation.

Stars

875

7-day growth

No data

Forks

38

Open issues

4

License

Apache-2.0

Last updated

2026-07-08

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It introduces a boundary-centric self-supervised objective that simultaneously captures semantic grouping and geometric structure, and provides a range of backbones up to 1.1B parameters that can be dropped into dense downstream tasks with minimal adaptation.

Who it is for

  • Computer vision researchers focusing on self-supervised learning
  • Practitioners building depth estimation or segmentation systems
  • Developers needing high-quality pretrained visual encoders
  • Researchers working on spatial perception and dense prediction

Use cases

  • Dense feature visualization with PCA projections
  • Monocular depth estimation and depth completion
  • Semantic segmentation with boundary-faithful features
  • Zero-shot video object segmentation via token matching

Strengths

  • Self-supervised pretraining with a novel boundary modeling objective that yields both semantic and spatial features
  • Multiple model sizes (ViT-S to ViT-g) with released weights on Hugging Face and ModelScope
  • Demonstrated effectiveness as a drop-in encoder for depth estimation (LingBot-Depth 2.0) and other dense tasks
  • Publicly available under Apache 2.0 license with straightforward installation and usage

Considerations

  • Reported performance gains rely on a large-scale 150M-sample RGB-D dataset not publicly released
  • Only backbone weights are provided; training code, projection heads, and boundary heads are not included
  • Large models (ViT-g) require significant GPU memory and compute resources for inference

README quick start

LingBot-Vision: Vision Pretraining for Dense Spatial Perception


🔭 Meet LingBot-Vision! A family of self-supervised ViT backbones for dense spatial perception! 🖼️📐

LingBot-Vision is a family of self-supervised Vision Transformer backbones for dense spatial perception, from ViT-S/16 up to a 1.1B-parameter ViT-g/16. The flagship model is pretrained with masked boundary modeling — a boundary-centric objective that encourages spatially structured patch features while retaining strong semantic representations.

Boundary-centric masked modeling. Each row shows the input image, the PCA projection of frozen patch tokens, teacher-discovered boundary tokens, and cosine-similarity maps from selected boundary-token queries. The features capture semantic grouping and geometric structure at the same time.

LingBot-Vision learns boundaries, shapes, and semantic regions all together, making it a drop-in visual encoder for dense downstream tasks:

  • 🎨 Dense feature visualization — PCA maps of frozen patch tokens reveal coherent object regions and crisp boundaries
  • 📏 Depth estimation — frozen patch tokens expose spatial structure to lightweight dense readouts
  • 🧩 Semantic segmentation — boundary-faithful features align region transitions with object contours
  • 🎬 Video object segmentation — training-free token matching and label propagation with frozen features
  • 🤖 Depth completion — LingBot-Vision is the visual encoder initialization for LingBot-Depth 2.0 (see below)

🌊 Meet LingBot-Depth 2.0

By simply replacing the encoder with LingBot-Vision at the ViT-L/16 and ViT-g/16 scales, and scaling the curated RGB-D training corpus from 3M to 150M samples, LingBot-Depth 2.0 achieves substantial performance gains over the previous and other system, as detailed in the technical report.

LingBot-Depth 2.0 on mirror and glass scenes. Each group shows input RGB, raw sensor depth, refined depth, and refined point clouds. Raw depth is missing on difficult surfaces such as window panes, glass balustrades, and reflective floors.

Description

Self-supervised learning for spatial perception

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo
Featured
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents
2,390
slvDev
Featured
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models
1,960
littledivy
Featured
littledivy GitHub avatar

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI & Machine Learning
1,482