GUIDE · 5 MIN READ
Humanoid locomotion
How humanoid locomotion controllers are trained, simulated, tracked and deployed, from parallel RL and sim-to-sim checks to whole-body control.
REVIEWED BY WBH · · DRAFTED WITH AI ASSISTANCE FROM THE CITED SOURCES
For: People who build, program or evaluate humanoid robots.
Learning to walk with massively parallel simulation
Rudin et al. present and study a training set-up that generates policies quickly for real-world robotic tasks, using massive parallelism on a single workstation GPU. [1] They pair it with a novel game-inspired curriculum that suits training with thousands of simulated robots in parallel. [1]
The authors report that flat-terrain policies train in under four minutes and uneven-terrain policies in twenty minutes, and they transfer the policies to the real robot to validate the approach. [1]
Note the robot: the paper trains the quadrupedal ANYmal, not a humanoid, so it is not a humanoid result. The authors also open-source their training code. [1]
From training to hardware: the sim-to-real workflow
Unitree's reinforcement-learning repository, which supports the Unitree Go2, H1, H1_2 and G1, lays out the basic workflow for motion control as Train, Play, Sim2Sim, Sim2Real. [2] Each stage answers a different question:
- Train: the robot interacts with the simulated environment to find a policy that maximises the designed rewards. The README advises against real-time visualisation during training because it reduces efficiency. [2]
- Play: the trained policy is replayed to verify that it meets expectations. [2]
- Sim2Sim: the Gym-trained policy is deployed to another simulator, to ensure it is not overly specific to the characteristics of Gym. [2]
- Sim2Real: the policy is deployed to the physical robot, which the README says must be in debug mode first. [2]
In this repository the second simulator is MuJoCo, and the repository acknowledges legged_gym as the foundation for its training code and rsl_rl for the reinforcement-learning algorithm implementation. [2]
Holosoma is a humanoid robotics framework from amazon-far. It covers training and deploying reinforcement-learning policies, supports locomotion (velocity tracking) and whole-body tracking tasks, and uses a shared inference pipeline across simulation and real robot control. [4] It lists Unitree G1 and Booster T1 as supported humanoids, and PPO and FastSAC as algorithms. [4]
What the simulator has to get right
MuJoCo is a general-purpose physics engine for articulated structures interacting with their environment. [3] Its models are written in the MJCF description language, and URDF files can also be loaded. [3]
In MuJoCo, a floating body is connected to the world by a free joint, and the position data of such a joint is 7 numbers (a 3D position and a unit quaternion) while the velocity data is 6 numbers. [3]
MuJoCo's contact model is soft by design, so gradual contact slip cannot be avoided, and the documentation says the solref and solimp parameters need adjusting to reduce it. [3] Stability and speed trade against each other through the time step: divergence is a hint that the timestep is too large for the chosen integrator. [3]
For large-scale training, MuJoCo has GPU-accelerated backends, MJX (JAX) and MuJoCo Warp (NVIDIA Warp). [3] Holosoma lists MuJoCo Warp as a training simulator, while plain MuJoCo is inference only there. [4]
From walking to whole-body control
NVIDIA's open repository for its whole-body control projects hosts model checkpoints and scripts for training, evaluating and deploying whole-body controllers for humanoid robots. [5] One of the controllers it supports is a decoupled design, with reinforcement learning for the lower body and inverse kinematics for the upper body. [5]
The same repository describes SONIC, which takes a different approach: it uses motion tracking as a scalable training task, so that a single unified policy produces whole-body movement across behaviours from walking and crawling to teleoperation. [5] Its released checkpoints are for the Unitree G1. [5]
The repository also documents a tested deployment tuning for one SONIC checkpoint that increases the control gains of the left and right ankle-pitch motors, which it says improves whole-body stability. [5]
The AMS paper (Agility Meets Stability) observes that recent methods have progressed in either agile dynamic skills or stability-critical behaviours, but remain specialised in one capability at the expense of the other. AMS unifies dynamic motion tracking and extreme balance maintenance in a single policy by combining human motion-capture data with synthetic balance motions and a hybrid reward scheme. [6] The authors validate it in simulation and on a real Unitree G1 humanoid. [6]
Adding perception: parkour
The Perceptive Humanoid Parkour paper composes retargeted atomic human skills into long-horizon trajectories by motion matching, formulated as nearest-neighbour search in a feature space. [7]
It then trains motion-tracking reinforcement-learning expert policies for these composed motions and distils them into a single depth-based, multi-skill student policy, using a combination of DAgger and reinforcement learning. [7] With only onboard depth sensing and a discrete 2D velocity command, the robot selects whether to step over, climb onto, vault or roll off obstacles. [7]
The authors validate the framework with extensive real-world experiments on a Unitree G1 humanoid robot, covering dynamic parkour skills and long-horizon traversal of several obstacles with closed-loop adaptation to real-time obstacle perturbations. [7]
Model-based software stacks
IHMC Open Robotics Software is an open-source stack that its README says has been used to power a wide variety of humanoid robots, including Nadia, Valkyrie and Alex. [8] It is licensed under the Apache 2.0 licence, builds with Gradle and requires JDK 17. [8]
The README also lists related IHMC libraries that the repository uses, including a rigid-body dynamics library (Mecano), a collection of algorithms for solving convex optimisation problems, soft real-time Java threading on Linux using RT_PREEMPT, and a Java EtherCAT master. [8] It additionally provides a ROS 2 API for many of its core components. [8]
Reading speed figures on spec sheets
Booster Robotics lists a walking speed of 1m/s and a turning speed of 1.5rad/s for the Booster T1, which has 23 degrees of freedom in total, 6 of them in each leg. [9]
EngineAI lists a moving speed of >2m/s for the EngineAI PM01, with the qualifier "Hardware Supported". [10]
When you read such a figure, check whether the source says what setting it applies to [6].