为什么值得关注

该代码库提供了一个灵活的框架，集成了最先进的视频生成模型（Wan2.2、Cosmos-Predict2）与多种动作建模范式，包含针对 LIBERO 和 RoboTwin 2.0 的预训练检查点，并在 RoboTwin 2.0 上实现了 89.5% 的高平均性能。

适合谁使用

机器人学习研究人员
具身智能从业者
探索动作预测的视频生成研究人员
基于世界模型构建机器人策略的开发者

典型使用场景

在 LIBERO 和 RoboTwin 基准上训练和评估 WAM
比较不同 WAM 家族（MoT、shared-DiT、特征条件）
扩展新的骨干适配器或动作模型变体
使用提供的预训练检查点复现研究结果

项目优势

模块化设计：通过类型化 YAML 配置系统（直接转为 Python 数据类）将骨干网络与 WAM 家族分离
支持多种 WAM 家族（mot_wam、shared_dit_wam、feature_conditioned_action_model），采用不同的注意力机制
包含两个主要基准（LIBERO、RoboTwin 2.0）的预训练检查点，并提供完整的 rollout 命令
与流行的视频生成骨干（Wan2.2、Cosmos-Predict2）集成，并提供清晰的适配器接口

使用前须知

早期研究版本，文档有限，路线图仍在进行中（例如缺少技术报告）
目前仅支持四种 WAM 家族和两种骨干适配器，更多基准和变体计划中
尚未提供正式的引用格式或 BibTeX 条目；依赖外部预训练模型可能增加部署复杂性

README 快速开始

StarWAM: A Generalizable Codebase for World-Action Models

StarWAM is a research codebase for building World-Action Models (WAMs): robot policies that combine generative video/world models with action prediction modules. It is designed for modular experimentation with world-model backbones, action representations, and training recipes.

This repository is an early research release. More WAM variants, benchmarks, model checkpoints, and technical details will be added.

News

2026/07: RoboTwin 2.0 integration — dual-arm MoT recipe, client/server rollout adapter for the official RoboTwin harness (89.5% avg over 50 tasks).
2026/07: Initial StarWAM codebase prepared with Wan2.2 and Cosmos-Predict2 backbone adapters, LIBERO training/rollout recipes, MoT WAM, Shared-DiT WAM, and feature-conditioned WAM support.

Highlights

World-model backbones: reuse pretrained video generation models as robot world models, e.g., Wan2.2 and Cosmos-Predict2.
Multiple WAM families:
- mot_wam: multi-stream video/action experts with mixed attention, e.g., Motus / FastWAM-style world-action modeling.
- shared_dit_wam: shared-DiT/register-token video-action prediction, e.g., DreamZero / LingBot-VA-style shared-token formulations.
- feature_conditioned_action_model: action prediction conditioned on video/world-model features, e.g., Video-IDM, Mimic-Video / World2Action, and StarVLA-WM4A-style variants.
Benchmark recipes: benchmark-specific data loading, normalization, text embedding caches, training recipes, and rollout utilities.
Typed recipe system: YAML recipes are loaded into Python dataclasses without requiring Hydra.

Repository Layout

starwam/                     # Core Python package
  backbone/                  # Wan2.2 / Cosmos-Predict2 backbone adapters
  wam/                       # WAM wrappers and taxonomy-level model families
  modules/                   # DiT blocks, MoT, scheduler, shared-DiT modules
  action_model/              # Action expert builders
  data/                      # LeRobot dataset and text-cache utilities
  training/                  # Trainer, losses, flow utilities, entrypoint
  tools/                     # Preprocessing utilities
  utils/                     # Checkpoint and config helpers
configs/                     # Accelerate / DeepSpeed configs
examples/

项目描述

A Generalizable Codebase for World-Action Models

StarWAM