This repository reproduces DreamWaQ's asymmetric actor-critic and CENet estimator in Isaac Lab on a Unitree Go2, enabling a proprioception-only policy to match a privileged baseline with minimal performance loss.
5
No data
0
0
MIT
2026-07-28
It demonstrates a clean single-phase training pipeline that replaces privileged terrain and velocity inputs with learned latent context from proprioceptive history, achieving near-privileged locomotion on real-world hardware constraints.
A reproduction of the two core contributions of DreamWaQ (Nahrendra, Yu & Myung, ICRA 2023) in NVIDIA Isaac Lab on a Unitree Go2: an asymmetric actor-critic and CENet, a context-aided estimator that lets the deployed policy run on proprioception alone.
Result: the proprioception-only policy matched a privileged baseline on terrain difficulty and episode length, trailing it by ~3 reward and ~0.1 m/s tracking error. CENet recovered true body velocity to MSE ≈ 0.006 (m/s)² from proprioceptive history alone.
https://github.com/user-attachments/assets/daabeb17-dc1e-4bb3-8c0d-82a0601c9b6a
Trained Go2 robots crossing discrete-obstacle terrain in Isaac Sim, each tracking its own velocity command.
In simulation an RL policy can see privileged information - the exact terrain under the robot, its true base velocity. A real quadruped has neither. It has joint encoders and an IMU. So a policy trained on privileged inputs cannot be deployed.
DreamWaQ's answer is to have the robot infer that hidden state from its own recent proprioceptive history, and to train the policy, the value function, and the estimator together in a single phase - no teacher-student distillation.
Asymmetric actor-critic. The actor sees only what hardware can measure (45-D proprioception) plus CENet's outputs. The critic, used only during training, additionally sees a 187-D terrain height scan and the true base velocity.
| Group | Dim | Contents | Seen by |
|---|---|---|---|
policy | 45 | ang. velocity, gravity, command, joint pos/vel, last action | actor + critic |
height_scan | 187 | 17×11 terrain height grid (privileged) | critic only |
privileged_vel | 3 | true base linear velocity | critic + CENet target |
Actor input 64 (45 + 3 velocity + 16 latent) · Critic input 235 (45 + 187 + 3)
CENet. A shared encoder over the last 5 proprioceptive frames (225-D) splits into two heads: a velocity head regressing 3-D body velocity, and a β-VAE head producing a 16-D latent context. Trained with a hybrid loss:
L = MSE(v̂, v) + MSE(ô, o) + β · KL( z ‖ N(0, I) )
CENet's outputs are detached before reaching the actor, so the policy's gradient cannot reshape the estimator - it learns only from its own prediction error.
Both policies: 2000 iterations, se
Reproduction of DreamWaQ (ICRA 2023) in NVIDIA Isaac Lab - asymmetric actor-critic + CENet for proprioception-only quadrupedal locomotion on a Unitree Go2
Similar projects matched by category, topics, and programming language.
Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.