Why it is worth attention

It unifies several state-of-the-art methods (SONIC, ADD, BPO, etc.) into a single configurable framework, provides pretrained policies and a terrain motion optimizer, and supports both Isaac Lab and MuJoCo backends.

Who it is for

Robotics RL researchers
Humanoid locomotion developers
Users of Isaac Lab simulation
Researchers interested in motion imitation and terrain adaptation

Use cases

Training motion-tracking policies for the Unitree G1 robot
Evaluating imitation learning algorithms on flat and uneven terrain
Generating terrain-adapted motion clips using the MPPI optimizer
Distilling vision-based policies from teacher networks

Strengths

Config-driven task composition with independent axis selection
Comprehensive set of pretrained checkpoints and demo motion data
Integration of multiple advanced algorithms (adversarial imitation, cross-embodiment, etc.)
Includes a terrain motion optimizer (TCRS) for adaptive clip generation

Considerations

Requires an existing Isaac Lab installation (not standalone)
Only supports the Unitree G1 humanoid robot out of the box
Terrain, vision, and distillation tasks are Isaac-Lab-only; MuJoCo backend has limited task support and needs fine-tuning for checkpoint transfer

README quick start

PMT — Perceptive Motion Tracking

Project page: https://acodedog.github.io/perceptive-bfm/

Cartwheel
Backflip

PMT trains humanoid motion-tracking RL policies for the Unitree G1 on Isaac Lab: DeepMimic-style imitation, vision/terrain perception, cross-embodiment (SONIC), adversarial imitation (ADD), and teacher → distill → finetune pipelines. Everything is config-driven: a task selects independent axes (robot / terrain / motion / obs / reward / network / algorithm / stage) and the builder derives the coupled ones, failing loud on invalid combos.

What's in this repo

Training & playback — scripts/train.py / scripts/play.py over a catalog of ~25 G1 tasks (Task catalog), config-composed from configs/ by the pmt_tasks/ task layer.
Pretrained policies — ready-to-run G1 checkpoints under checkpoints/pretrained/.
Demo motion data — a 99-pair sample of raw/optimized terrain clips + terrain meshes under assets/ so the terrain/distill tasks run out-of-the-box.
TCRS — an MPPI terrain-motion optimizer (TCRS/) that generates the optimized clips from raw motion + a terrain scene.
MuJoCo backend — the flat task family also runs on mjlab (MuJoCo-Warp); see docs/MJLAB_USAGE.md.

Scope: this is a research/training codebase. It runs inside an existing Isaac Lab install (it does not vendor Isaac Sim / Isaac Lab) — see Install.

Install

PMT runs inside an Isaac Lab Python environment — it does not vendor Isaac Sim / Isaac Lab. Get a working Isaac Lab install first, then install PMT (editable) into that env:

conda activate            # the env that has Isaac Lab + Isaac Sim
export OMNI_KIT_ACCEPT_EULA=YES          # accept the Omniverse EULA for headless launches
cd /path/to/PMT
python -m pip install -e .               # installs the motion_tracking_rl RL core

Run all commands from the repository root (pmt_tasks/, configs/, scripts/ are used in place).

Prerequisites: an importable isaaclab / isaaclab_tasks, Python ≥ 3.10, the Unitree G1 robot assets (resolved via PMT_ASSET_DIR — not vendored), and per-task motion *.npz clips.

**Pat

Description

Official implementation of Perceptive Behavior Foundation Model

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo

Featured

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents

2,390

slvDev

Featured

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models

1,960

littledivy

Featured

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI & Machine Learning

1,482