Walking and running is notoriously difficult to recreate in robots. Now, a group of researchers has overcome some of these challenges by creating an innovative method that employs central pattern generators — neural circuits located in the spinal cord that generate rhythmic patterns of muscle activity — with deep reinforcement learning. An international group of researchers has created a new approach to imitating human motion through combining central pattern generators (CPGs) and deep reinforcement learning (DRL). The method not only imitates walking and running motions but also generates movements for frequencies where motion data is absent, enables smooth transition movements from walking to running, and allows for adapting to environments with unstable surfaces.
Details of their breakthrough were published in the journal IEEE Robotics and Automation Letters on April 15, 2024.
We might not think about it much, but walking and running involves inherent biological redundancies that enable us to adjust to the environment or alter our walking/running speed. Given the intricacy and complexity of this, reproducing these human-like movements in robots is notoriously challenging.
Current models often struggle to accommodate unknown or challenging environments, which makes them less efficient and effective. This is because AI is suited for generating one or a small number of correct solutions. With living organisms and their motion, there isn’t just one correct pattern to follow. There’s a whole range of possible movements, and it is not always clear which one is the best or most efficient.
DRL is one way researchers have sought to overcome this. DRL extends traditional reinforcement learning by leveraging deep neural networks to handle more complex tasks and learn directly from raw sensory inputs, enabling more flexible and powerful learning capabilities. Its disadvantage is the huge computational cost of exploring vast input space, especially when the system has a high degree of freedom.
Another approach is imitation learning, in which a robot learns by imitating motion measurement data from a human performing the same motion task. Although imitation learning is good at learning on stable environments, it struggles when faced with new situations or environments it hasn’t encountered during training. Its ability to modify and navigate effectively becomes constrained by the narrow scope of its learned behaviors.
“We overcame many of the limitations of these two approaches by combining them,” explains Mitsuhiro Hayashibe, a professor at Tohoku University’s Graduate School of Engineering. “Imitation learning was used to train a CPG-like controller, and, instead of applying deep learning to the CPGs itself, we applied it to a form of a reflex neural network that supported the CPGs.”
CPGs are neural circuits located in the spinal cord that, like a biological conductor, generate rhythmic patterns of muscle activity. In animals, a reflex circuit works in tandem with CPGs to provide adequate feedback that allows them to adjust their speed and walking/running movements to suit the terrain.
By adopting the structure of CPG and its reflexive counterpart, the adaptive imitated CPG (AI-CPG) method achieves remarkable adaptability and stability in motion generation while imitating human motion.
“This breakthrough sets a new benchmark in generating human-like movement in robotics, with unprecedented environmental adaptation capability,” adds Hayashibe “Our method represents a significant step forward in the development of generative AI technologies for robot control, with potential applications across various industries.”
The research group comprised members from Tohoku University’s Graduate School of Engineering and the École Polytechnique Fédérale de Lausanne, or the Swiss Federal Institute of Technology in Lausanne.