WU-CVGL/NeuWorld - 仓库详情

AI 仓库情报摘要

FR-AI / ANALYSIS

为什么值得关注

它引入了以场景为中心的 rollout 和分解交互范式，用固定长度的可渲染神经隐式场景状态取代不断增长的视频隐式轨迹，实现了统一的条件输入和无需预训练视频骨干或辅助三维重建器的从头训练，并被 ECCV 2026 接收。

适合谁使用

研究神经场景表示的计算机视觉学者
自动驾驶仿真工程师
需要交互式场景探索的机器人研究人员
关注可控世界生成的游戏开发者

典型使用场景

通过相机位姿控制的交互式三维场景探索
具有回访一致性的长程导航仿真
自动驾驶感知系统的数据增强
为具身 AI 代理生成虚拟环境

项目优势

以场景为中心的 rollout 避免了轨迹复杂度增长
分解交互将隐式状态转换与位姿条件渲染分离
统一的 NIS 条件将所有输入映射到同一模态，减少编码器异构性
在公开数据集上从头训练，无需依赖预训练视频骨干

使用前须知

代码和模型权重尚未公开（仍在内部审查中）
新颖的架构可能需要非专业用户进行大量适配
神经隐式渲染的计算成本可能较高，难以满足实时应用

README 快速开始

Walking in the Implicit: Interactive World Exploration via Neural Scene Representation

Code will be released soon.

Zhiqi Li1,2 Chengrui Dong1,2 Zhenhua Du1,2 Hangning Zhou3,† Cong Qiu3 Hailong Qin3 Mu Yang3 Dongxu Wei2 Peidong Liu2,*

1Zhejiang University 2Westlake University 3Afari Intelligent Drive †Project Lead *Corresponding Author

News

The public repository is under internal review. Code and checkpoints will be released soon.
🎉🎉 NeuWorld is accepted by ECCV 2026.

Highlights

Scene-centric rollout. We replace growing video-latent trajectories with a fixed-length, renderable Neural Implicit Scene (NIS) state.
Factorized interaction. Each step decouples stochastic latent scene-state transition from deterministic pose-conditioned rendering.
Unified NIS conditioning. Camera, reference-image, and retrieved history cues are mapped into the same NIS modality instead of separate heterogeneous encoders.
Long-horizon consistency. NeuWorld is designed for camera-controlled exploration with revisitation consistency and favorable inference efficiency.
From-scratch training. The model is trained on public posed-view datasets without pretrained video backbones or auxiliary 3D reconstructors.

Method Overview

At each interaction step, the frozen NIS-VAE encoder maps the current observation and a sparse future pose trajectory to a partial NIS condition. Geometry-aware retrieval selects a history set and encodes it as memory NIS tokens. NIS-DiT samples the next local NIS state, and the frozen decoder renders future views under the queried poses.

Citation

If you find our work useful, please cite:

@inproceedings{li2026neuworld,
  title     = {Walking in the Implicit: Interactive World Exploration via Neural Scene Representation},
  author    = {Li, Zhiqi and Dong, Chengrui and Du, Zhenhua and Zhou, Hangning and Qiu, Cong and Qin, Hailong and Yang, Mu and Wei, Dongxu and Liu, Peidong},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}

项目描述

[ECCV 2026] Walking in the Implicit: Interactive World Exploration via Neural Scene Representation

NeuWorld