End-to-End High-Level Control of Lower-Limb Exoskeleton for Human Performance Augmentation Based on Deep Reinforcement Learning

Abstract

This paper proposes a novel end-to-end controller for the lower-limb exoskeleton for human performance augmentation (LEHPA) systems based on deep reinforcement learning (E2EDRL). The model-free controller contains two control levels: the high-level control responsible for end-to-end human motion intention recognition based on the exoskeleton state signals and the human-exoskeleton interaction (HEI) force signals by deep neural network predictor, and the lowlevel control for motion tracking by joint PD controllers. The deep neural network predictor does not require complex kinematic calculations that are inevitable in conventional human motion intention recognition methods. We execute the learning process in simulation to learn the E2EDRL strategy efficiently and safely by constructing a novel multibody simulation environment and proposing its specific hybrid inverse-forward dynamics simulation method. The passive mode (all joints remain unpowered) is introduced as a benchmark for comparison purposes. A novel performance assessment method based on HEI forces is put forward to evaluate the E2EDRL strategy quantitatively. The global ratio of the HEI forces in the E2EDRL strategy relative to those in the passive mode is as low as 0.65. The global reduction of the HEI forces demonstrates the superior control performance of the E2EDRL strategy.INDEX TERMS Lower-limb exoskeleton for human performance augmentation, end-to-end human motion intention recognition, deep reinforcement learning

Advancements in Sensor Technologies and Control Strategies for Lower-Limb Rehabilitation Exoskeletons: A Comprehensive Review

“…Data-driven models, such as deep reinforcement learning, adapt to the user's walking style and predict gait patterns, providing personalized assistance and optimizing energy efficiency in real-time [88]. Non-model control is a superior alternative to modelbased approaches as it directly manipulates control inputs based on sensor feedback [89]. Assist-as-needed control expertly adjusts the level of assistance based on the user's needs, promoting active participation in movement tasks [6].…”

“…However, challenges such as the need for large training datasets and the transferability of learned behaviors to real-world settings remain. Zheng et al [89] introduced a deep reinforcement learning framework for developing a model-free walking controller for lower-limb exoskeletons, aiming to enhance human performance augmentation. The controller, based on a deep neural network, directly estimates human motion intentions without the need for a kinematic or dynamic model of the exoskeleton system.…”

“…To address these challenges, a non-model-based control approach, incorporating nonlinear feedback control, was used to overcome model disturbances, thus enhancing the system's robustness. Zheng et al [89] proposed an end-to-end controller for a lower-limb exoskeleton system using deep reinforcement learning (E2EDRL). This controller consisted of high-level control for recognizing human motion intentions and low-level control for motion tracking, eliminating the reliance on complex kinematic or dynamic models.…”

Adaptive Gait Generation Using Dynamic Movement Primitives with Multimodal Sensors for Assistive Lower-limb Exoskeletons

“…Another exoskeleton was designed to facilitate activities of daily living through a novel locomotion mode recognition method, increasing user autonomy [6]. Additionally, in [7], an exoskeleton aimed for human augmentation assists in walking, reflecting a trend towards enhancing daily life assistance levels in exoskeleton research.…”

A Novel Locomotion Rule Rmbedding Long Short-Term Memory Network with Attention for Human Locomotor Intent Classification Using Multi-Sensors Signals

“…Recently, deep reinforcement learning (DRL) has garnered significant attention an interactive ML paradigm that seamlessly integrates deep neural networks (DNNs) [16] into the well-established conventional reinforcement learning (RL) framework [17]. Many DRL approaches, such as the Markov decision process (MDP) and self-organizing networks have obtained remarkable achievements in several fields [18][19][20], offering new possibilities and advancements in the field of lowerlimb prosthetics. However, RL typically requires a large number of interaction samples for training, which can be a challenge in the real world, it still needs further development before being applied to practical scenarios.…”