Self Gradient Forcing (SGF) enables video diffusion models trained on only 5-second clips to generate minute-long videos with improved identity, layout, and temporal consistency by recovering the missing context-gradient path via a bounded two-pass replay mechanism.
106
No data
2
1
Apache-2.0
2026-07-23
It directly tackles the long-video extrapolation challenge—a major bottleneck in current video generation—without needing longer training sequences, and provides open-source code, pretrained models, and clear inference/training instructions built on top of the Wan video model ecosystem.
Junhao Zhuang, Shiyi Zhang, Yuxuan Bian, Yaowei Li, Yawen Luo, Yijun Liu, Weiyang Jin, Songchun Zhang, Xianglong He, Xuying Zhang, Haoran Li, Haoyang Huang, Zeyue Xue, Nan Duan
Joy Future Academy, JD
⭐ If Self Gradient Forcing is useful for your research, please consider starring this repository.
Self Gradient Forcing (SGF) recovers the missing context-gradient path for self-generated causal memory through a bounded two-pass replay, enabling models trained with only a 5-second window to extrapolate to minute-scale videos with stronger identity, layout, and temporal stability.
The environment follows the Causal-Forcing setup.
conda create -n self_gradient_forcing python=3.10 -y
conda activate self_gradient_forcing
pip install -r requirements.txt
pip install flash-attn --no-build-isolation
python setup.py develop
bash scripts/download_weights.sh
The script uses the Hugging Face CLI command hf by default. Set HF_CLI=huggingface-cli if your environment still uses the older command name.
It downloads:
wan_models/Wan2.1-T2V-1.3B and wan_models/Wan2.1-T2V-14B.checkpoints/init/framewise/ and checkpoints/init/chunkwise/: ar_diffusion.pt, causal_cd.pt, and causal_ode.pt.checkpoints/framewise/ar/model.pt and checkpoints/chunkwise/ar/model.pt.prompts/vidprom_filtered_extended.txt.The default prompt file is prompts/test_prompt.txt with 8 prompts. The launcher uses 8 GPUs when at least 8 GPUs are visible; otherwise it falls back to single-GPU serial inference. By default it generates 963 latent frames, which decode to about 240 seconds of video at 16 fps.
The inference script takes the release setting name (framewise or chunkwise) and selects the matching config and checkpoint automatically:
configs/self_gradient_forcing_framewise.yamlSelf Gradient Forcing (SGF) recovers the missing context-gradient path for self-generated causal memory through a bounded two-pass replay, enabling models trained with only a 5-second window to extrapolate to minute-scale videos with stronger identity, layout, and temporal stability.
Similar projects matched by category, topics, and programming language.
Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.