A Lower Limb Exoskeleton Adaptive Control Method Based on Model-free Reinforcement Learning and Improved Dynamic Movement Primitives

Recent advancements in lower limb exoskeleton control have predominantly focused on enhancing walking capabilities across diverse terrains, such as level ground, stairs, and ramps. However, achieving seamless transitions between these terrains remains a significant challenge due to the unpredictability of the environment, which hampers adaptive control. In this paper, we propose a Hierarchical Interactive Learning (HIL) control method based on gait phase and locomotion pattern recognition. The method comprises two layers: high-level learning and low-level control. The high-level learning is based on gait phase and locomotion pattern recognition, utilizing the Dynamic Movement Primitives (DMP) to piecewise learn the desired joint torque curves. The low-level control utilizes the learned DMP to output torque based on the gait phase and locomotion pattern, while reinforcement learning is employed to dynamically adjust the control parameters of DMP in real-time with the goal of minimizing human-exoskeleton interaction forces. The experiments collected gait data of lower limb movement from active exoskeletons. The results show that our method significantly reduces human-exoskeleton interaction forces across diverse terrains. In order to verify the feasibility and effectiveness of the proposed method, 15 healthy subjects were tested with the lower limb exoskeleton of these 3 generations. The experimental results show that the proposed HIL control method gives a valuable tool for smooth transitions among different terrains, reduces the reliance on accurate dynamic models and the average oxygen consumption decreased by about 12%, underscoring its potential to improve exoskeleton-assisted mobility.