Until now, all robots, no matter how sophisticated, have moved stiffly and mechanically. But new motion-tracking technology powered by artificial intelligence is changing that.
A recent video clip shows a Unitree G1 robot dancing, sparring, and exercising beside a human companion. The robot can even mimic its partner’s motions in real-time.
Step one: gather motion data
This dramatic display is thanks less to the robot itself than to its new, advanced software. ExBody2 is a new technology that was developed by UC San Diego, Berkeley, MIT, and Nvidia. It takes detailed scans and motion-tracked visualizations of human movements and uses them to train humanoid robots. After gathering the motion data, ExBody2 translates it into a format which the computer-brains of robots can read.
It uses what’s called “reinforcement learning ” to do this. This involves feeding a model a large amount of data and refining it through repeated trial and error. Ideally, this creates a model capable of making decisions that maximize positive outcomes.
The robot follows the motions of a human researcher. Photo: ExBody2
ExBody2 currently uses a dataset of mostly upper-body movements because the robot’s lower-body range of movement is more limited.
Even without much lower body input, however, the dataset included over 2,800 movements. It drew heavily from a commercial collection of human movements called Archive of Motion Capture As Surface Shapes (AMASS).
Digital models of the movement, next to the real-world manifestation. Photo: Li et al.
The next dance
The need to hand curate the collection is a huge limitation to progress. Researchers hope that in the future, they can use other forms of AI to automate data collection and generate motion data.
One impressive aspect of the new technology is the real-time mimicry. The code, developed in China in 2023, is called HyberIK, short for Hybrid Analytical-Neural Inverse Kinematics for Body Mesh Recovery. Articulated hands and a realistic, expressive face enhance the realism.
The end goal, of course, is not to make a robot that dances. The motion technology will allow robots to perform a wider variety of jobs, including those requiring delicate, precise movements.
“This means that whatever humans can do, the robot can potentially learn,” says Xuanbin Peng, a University of California San Diego researcher involved in the project.
It’s too soon to say what the specific practical applications might be. For now, however, the robots seem content to waltz.
