Sensitivity Adaptation of Lower-Limb Exoskeleton for Human Performance Augmentation Based on Deep Reinforcement Learning

Abstract

The lower-limb exoskeleton for human performance augmentation (LEHPA) in sensitivity amplification control (SAC) is vulnerable to model parameter uncertainties and unmodeled dynamics due to its large sensitivity to external disturbances. This paper proposes sensitivity adaptation based on deep reinforcement learning (SADRL) to reduce the dependence on the model accuracy and tackle the ever-changing human-exoskeleton interaction (HEI) dynamics by interpreting the sensitivity adjustment as a Markov Decision Process (MDP). The agent learns an appropriate sensitivity for each exoskeleton state. To train the control policy safely and efficiently, a multibody simulation environment is created to implement the training process, accompanied by a novel hybrid inverse-forward dynamics simulation method to carry out the simulation. For comparison purposes, the SAC controller is introduced as a benchmark. A novel performance evaluation method based on the HEI forces at the back, thighs, and shanks is proposed to evaluate the control effect of the trained SADRL controller quantitatively. The SADRL controller is compared with the SAC controller at five specified walking speeds, resulting in a lumped HEI force ratio of 0.54. The total decrease of HEI forces demonstrates the superior control effect of the SADRL strategy.

Advancements in Sensor Technologies and Control Strategies for Lower-Limb Rehabilitation Exoskeletons: A Comprehensive Review

“…Slidingmode control ensures precise joint-angle control, providing stability and robustness to uncertainties and disturbances, resulting in smoother motion and an improved user experience [87]. Data-driven models, such as deep reinforcement learning, adapt to the user's walking style and predict gait patterns, providing personalized assistance and optimizing energy efficiency in real-time [88]. Non-model control is a superior alternative to modelbased approaches as it directly manipulates control inputs based on sensor feedback [89].…”

“…This control method combines two exoskeleton training modes, facilitating individual adaptation and active compliance in rehabilitation training [51]. Zheng et al [88] introduced a novel strategy, SADRL, combining sensitivity amplification control (SAC) with deep reinforcement learning (DRL), to enhance lower-limb exoskeleton control. Compared to SAC alone, SADRL demonstrates superior adaptability and control effectiveness, evidenced by a significant reduction in human-exoskeleton interaction forces.…”