LingBot-Vision 是一个自监督视觉Transformer骨干网络家族,采用掩码边界建模进行预训练,专注于深度估计和分割等密集空间感知任务。

Stars

875

7 天增长

暂无数据

Fork 数

38

开放 Issue

4

开源协议

Apache-2.0

最近更新

2026-07-08

AI 仓库情报摘要
FR-AI / ANALYSIS

为什么值得关注

它提出了一种以边界为中心的自监督目标,能够同时捕捉语义分组和几何结构,并提供了多达11亿参数的一系列骨干网络,可直接应用于密集下游任务而无需大量调整。

适合谁使用

  • 从事自监督学习的计算机视觉研究人员
  • 构建深度估计或分割系统的实践者
  • 需要高质量预训练视觉编码器的开发者
  • 研究空间感知和密集预测的学者

典型使用场景

  • 通过PCA投影进行密集特征可视化
  • 单目深度估计与深度补全
  • 具有边界忠实特征的语义分割
  • 基于令牌匹配的零样本视频对象分割

项目优势

  • 采用新颖的边界建模自监督预训练目标,同时获得语义与空间特征
  • 提供多种模型尺寸(ViT-S到ViT-g),权重已在Hugging Face和ModelScope发布
  • 作为深度估计(LingBot-Depth 2.0)等密集任务的即插即用编码器,展示了有效性
  • 基于Apache 2.0开源许可,安装和使用简便

使用前须知

  • 报告中性能提升依赖于未公开的1.5亿样本RGB-D数据集
  • 仅提供骨干网络权重,不包括训练代码、投影头部或边界头部
  • 大型模型(ViT-g)在推理时需要大量GPU内存和计算资源

README 快速开始

LingBot-Vision: Vision Pretraining for Dense Spatial Perception


🔭 Meet LingBot-Vision! A family of self-supervised ViT backbones for dense spatial perception! 🖼️📐

LingBot-Vision is a family of self-supervised Vision Transformer backbones for dense spatial perception, from ViT-S/16 up to a 1.1B-parameter ViT-g/16. The flagship model is pretrained with masked boundary modeling — a boundary-centric objective that encourages spatially structured patch features while retaining strong semantic representations.

Boundary-centric masked modeling. Each row shows the input image, the PCA projection of frozen patch tokens, teacher-discovered boundary tokens, and cosine-similarity maps from selected boundary-token queries. The features capture semantic grouping and geometric structure at the same time.

LingBot-Vision learns boundaries, shapes, and semantic regions all together, making it a drop-in visual encoder for dense downstream tasks:

  • 🎨 Dense feature visualization — PCA maps of frozen patch tokens reveal coherent object regions and crisp boundaries
  • 📏 Depth estimation — frozen patch tokens expose spatial structure to lightweight dense readouts
  • 🧩 Semantic segmentation — boundary-faithful features align region transitions with object contours
  • 🎬 Video object segmentation — training-free token matching and label propagation with frozen features
  • 🤖 Depth completion — LingBot-Vision is the visual encoder initialization for LingBot-Depth 2.0 (see below)

🌊 Meet LingBot-Depth 2.0

By simply replacing the encoder with LingBot-Vision at the ViT-L/16 and ViT-g/16 scales, and scaling the curated RGB-D training corpus from 3M to 150M samples, LingBot-Depth 2.0 achieves substantial performance gains over the previous and other system, as detailed in the technical report.

LingBot-Depth 2.0 on mirror and glass scenes. Each group shows input RGB, raw sensor depth, refined depth, and refined point clouds. Raw depth is missing on difficult surfaces such as window panes, glass balustrades, and reflective floors.

项目描述

Self-supervised learning for spatial perception

相关仓库与替代方案

根据分类、Topic 和编程语言匹配的相似项目。

lopopolo
精选
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI 与机器学习AI 智能体
2,390
slvDev
精选
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI 与机器学习大语言模型
1,960
littledivy
精选
littledivy GitHub avatar

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI 与机器学习
1,482