ShadowDancer is a research project that teaches interactive video world models any-action, frame-level control by learning unified dynamics representations from a video and its shadow.

Stars

9

7-day growth

No data

Forks

0

Open issues

0

License

No data

Last updated

2026-07-31

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It targets a key limitation of current video world models—restricted or predefined action spaces—by aiming for arbitrary actions and fine-grained control; the project page and arXiv paper make the approach publicly accessible while code release is planned.

Who it is for

  • Researchers in video generation and world models
  • Computer vision researchers studying action-conditioned dynamics
  • AI developers building interactive simulation or media tools

Use cases

  • Enabling frame-level user control in interactive video generation
  • Learning action representations for world models from videos without manual action labels
  • Building video-based simulation environments with arbitrary action inputs

Strengths

  • Proposes any-action, frame-level control for interactive video world models
  • Introduces a unified dynamics representation learned from a video and its shadow
  • Provides an arXiv paper and project website with visual teaser
  • States code and model weights are planned for release after internal review

Considerations

  • Code and model weights are not yet publicly available
  • README does not include benchmark numbers or system-level comparisons
  • Adoption currently depends on reading the paper and waiting for the official release

README quick start

 ShadowDancer

Teaching Video World Models Any Action from a Video and Its Shadow

ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadow

Jin Cao, Zian Meng, Kaipeng Zhang† († corresponding author)

ShadowDancer gives interactive video world models an any-action, frame-level control.

The code and model weights are being cleaned up and are undergoing internal review before release.🚧

Citation

@misc{cao2026shadow,
  title={ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadow},
  author={Jin Cao and Zian Meng and Kaipeng Zhang},
  year={2026},
  eprint={2607.28362},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2607.28362},
}

Description

Code release for https://shadowdancer-1.github.io/

Related repositories

Similar projects matched by category, topics, and programming language.

MoonshotAI
Featured
MoonshotAI GitHub avatar

Kimi-K3

Kimi K3 is an open-weight, 2.8T-parameter native multimodal agentic model with a 1M-token context window, designed for frontier coding, knowledge work, and reasoning tasks.

AI & Machine LearningAI Agents
3,348
xuchonglang
Featured
xuchonglang GitHub avatar

investing-for-beginners

A structured investing guide for Chinese beginners covering US stocks, options, and cryptocurrency, with focus on foundational concepts and risk awareness.

Blockchain & Web3
2,739
Krishnagangwal
Featured
Krishnagangwal GitHub avatar

CS-Fundamentals

A curated collection of Computer Science fundamentals (PDFs, notes, cheatsheets, interview question banks) for placement preparation, covering seven core subjects plus general resources.

Data & DatabasesDatabases & Storage
2,326