xiejunjie524
Featured
handdraw-story-video
A tool that converts 7–9 hand-drawn story keyframes into a 35–45 second vertical video with progressive line art and color animation, configurable via JSON and renders with HyperFrames and GSAP.
AI repository intelligence
FR-AI / ANALYSISWhy it is worth attention
It explicitly exploits temporal structure in video to improve SfM robustness and accuracy, achieving high benchmark scores (e.g., LaMAR WATE-AUC 90.22%) and is built on top of COLMAP with a configurable pipeline, making it accessible to practitioners.
Who it is for
- Computer vision researchers working on SfM, 3D reconstruction, or visual SLAM
- Robotics engineers needing accurate offline camera pose estimation from video
- AR/VR developers requiring sparse 3D maps from video sequences
- Benchmarking enthusiasts comparing video-based SfM methods
Use cases
- Reconstructing a sparse 3D map and camera trajectory from a handheld or drone video
- Evaluating and comparing video SfM algorithms on standard benchmarks (LaMAR, EuRoC, ETH3D-SLAM, CroCoDL)
- Generating high-quality reconstruction visualizations for debugging or presentation
- Building a localization map for subsequent visual-inertial or pure-visual navigation
Strengths
- Leverages temporal coherence (tracks, loop closures) and metric depth to produce accurate and robust reconstructions
- Achieves state-of-the-art results on multiple video SfM benchmarks as reported in the README
- Full-featured pipeline with Hydra-based configuration, allowing easy tuning of frontend and mapping parameters
- Includes built-in visualization tools (Rerun playback, interactive HTML) for understanding the optimization process and results
Considerations
- Requires a specific software stack (Linux x86-64, COLMAP 4.1, PyTorch, xFormers) and an NVIDIA GPU
- First run downloads ~9 GB of model checkpoints, and dataset preparation can require tens of GB of disk space
- Operates offline and is not designed for real-time or incremental SfM use cases
README quick start
VidMap
Exploiting Temporal Structure for Video-Based Structure-from-Motion
Zador Pataki · Paul-Edouard Sarlin · Marc Pollefeys
ECCV 2026
Paper
VidMap is an offline Structure-from-Motion system for video. It combines temporal tracks, loop closures, metric depth, and global optimization to estimate camera poses, camera intrinsics, and a sparse 3D map.
Setup
We provide the Python package vidmap. From a clean Python environment,
clone VidMap and its pinned model dependencies:
git clone --recursive https://github.com/cvg/vidmap.git && cd vidmap
Build and install COLMAP 4.1 and its PyCOLMAP bindings from source. Then install PyTorch, xFormers, and VidMap:
pip install -e .
VidMap was last tested on Linux x86-64 with Python 3.10, COLMAP and PyCOLMAP 4.1, PyTorch 2.7.1, TorchVision 0.22.1, and xFormers 0.0.31 on NVIDIA GPUs.
The first frontend run automatically downloads approximately 9 GB of model checkpoints.
Optional visualization setup. Install Rerun:
pip install rerun-sdk
Execution
The repository includes a silent example video. Run the frontend to build its mapper inputs, then run mapping:
INPUT_DATA=./assets/example_video.mp4
OUTPUT_DIR=./example_output/
MAPPER_INPUTS="$OUTPUT_DIR/mapper_inputs"
python -m vidmap.frontend "$INPUT_DATA" \
--output "$OUTPUT_DIR"
python -m vidmap.map \
--mapper-inputs "$MAPPER_INPUTS" \
--output "$OUTPUT_DIR"
Alternatively, run frontend and mapping together:
python -m vidmap.run \
--input_data "$INPUT_DATA" \
--output "$OUTPUT_DIR"
Alternatively, --input_data may point to a directory of extracted frames.
Both mapping commands write the final COLMAP model under $OUTPUT_DIR/rec.
Resolved frontend_config.yaml and mapping_config.yaml are written directly
under $OUTPUT_DIR.
Visualization
Repeat the reconstruction with visualization artifacts enabled:
python -m vidmap.map \
--mapper-inputs "$MAPPER_INPUTS" \
--output "$OUTPUT_DIR" \
--save-playback-trace \
--save-3d-html
--save-playback-trace records optimization states for Rerun playback.
`--save-
Description
VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion (ECCV 2026)
Related repositories
Similar projects matched by category, topics, and programming language.
bytec-ai
Featured
book-video-factory
A portable multi-account short video production skill for books that manages accounts, scripts, images, voiceovers, subtitles, and final exports through a unified workspace.
Krishnagangwal
Featured
CS-Fundamentals
A curated collection of Computer Science fundamentals (PDFs, notes, cheatsheets, interview question banks) for placement preparation, covering seven core subjects plus general resources.