Toward Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control

Li, Minhan; Wen, Yue; Gao, Xiang; Si, Jennie; Huang, He

doi:10.1109/tro.2021.3078317

Cited by 44 publications

(32 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such a gait pattern change with visual FB challenged the prosthesis tuning policy (learned when visual FB was off) to quickly accomplish the tuning task. Although we previously showed that our RL-learned policy for prosthesis control is robust and can be generalized to other gait patterns (10), additional iterations and time to converge was needed when the dynamics drastically change. Therefore, we postulate that adding a human control goal (i.e., gait timing control in this study) that aligns with the machine tuning goal may improve prosthesis tuning speed if the added human control goal does not signi cantly change the overall gait dynamics and pattern of the prosthesis user.…”

Section: Discussionmentioning

confidence: 99%

“…1. A reinforcement learning-based framework developed by our team (10) was applied to auto tune the impedance control parameters (see Prosthesis Tuning section for details). Participants were instructed to walk as consistently as possible on a treadmill while the control parameters updated every four gait cycles until the knee angle pro le was within the bound.…”

Section: Experimental Protocolmentioning

confidence: 99%

“…The adjustment was realized by an existing reinforcement learning algorithm as detailed in Li et. al (10), and it was considered a success when the tuning convergence criteria were met, where the peak and duration errors between the measured and normative knee angle pro les (as de ned in Fig. 1) were within 2° and 3% (i.e., target range), respectively, for 8 out of 10 consecutive iterations.…”

Section: Prosthesis Tuningmentioning

confidence: 99%

“…Further, the tuning procedure can become impractical with the increased number of control parameters needed to be tuned. Research groups have created and investigated their own automatic prosthesis tuning algorithms that adjust the control parameters (either joint impedance or torque pro les) of the prosthesis as the user is walking to achieve a pre-determined goal (8)(9)(10)(11). Focusing on transfemoral prostheses, our group designed model-free reinforcement learning (RL) algorithms to automatically tune 12 prosthesis control parameters to achieve normative knee motion in walking (10,11), which has been a common tuning goal for research and commercial robotic prostheses (4,5).…”

Section: Introductionmentioning

confidence: 99%

“…Research groups have created and investigated their own automatic prosthesis tuning algorithms that adjust the control parameters (either joint impedance or torque pro les) of the prosthesis as the user is walking to achieve a pre-determined goal (8)(9)(10)(11). Focusing on transfemoral prostheses, our group designed model-free reinforcement learning (RL) algorithms to automatically tune 12 prosthesis control parameters to achieve normative knee motion in walking (10,11), which has been a common tuning goal for research and commercial robotic prostheses (4,5). Through iterative prosthesis parameter tuning, these auto-tuning algorithms can successfully yield prosthesis kinematics, approximating the features of desired knee pro le (including joint peak values and timing) in gait within an error bound.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Human-Prosthesis Cooperation: Combining Adaptive Prosthesis Control with Visual Feedback Guided Gait

Fylstra

Lee

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Background: Personalizing prosthesis control is often structured as human-in-the-loop optimization. However, gait performance is influenced by both human control and intelligent prosthesis control. Hence, we need to consider both human and prosthesis control to achieve desired gait patterns. We developed a novel paradigm that engages human gait control via visual feedback (FB) of stance time to cooperate with automatic prosthesis tuning. Three questions were studied: 1.) does user control of gait timing (via visual FB) help the prosthesis tuning algorithm to converge faster? 2.) in turn, does the prosthesis control influence the user’s ability to reach and maintain the feedback goal? and 3.) does the prosthesis control parameters tuned with extended stance time on prosthesis side allow the user to maintain this potentially beneficial behavior even after feedback is removed (short- and long-term retention)?Methods: A reinforcement learning algorithm was used to achieve prosthesis control to meet normative knee kinematics in walking. A visual FB system was used to cue human control of stance time to facilitate the prosthesis tuning goal. Seven individuals without amputation (ND) and four individuals with transfemoral amputation (TFA) walked with a powered knee prosthesis on a treadmill. Participants completed prosthesis autotuning with three visual feedback conditions: no FB, self-selected stance time FB (SS FB), and increased stance time FB (Inc FB). The retention of FB effects was studied by comparing the gait performance across three different prosthesis controls, tuned with different visual FB.Results: 1) Human control of gait timing reduced the tuning duration in individuals without amputation, but not for individuals with TFA. 2) The change of prosthesis control did not influence users’ ability to reach and maintain the visual FB goal. 3) All participants increased their prosthesis-side stance time with the feedback and maintain it right after feedback was removed. However, in the post-test, the tuned prosthesis control parameters with visual FB only supported a few participants with longer stance time and better stance time symmetry. Conclusions: The study gained insights on human-prosthesis cooperation, which may guide the future successful adoption of this paradigm in prosthesis control personalization or human-in-the-loop optimization for improved gait performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Experimental Protocolmentioning

confidence: 99%

Section: Prosthesis Tuningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Human-Prosthesis Cooperation: Combining Adaptive Prosthesis Control with Visual Feedback Guided Gait

Fylstra

Lee

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Multibody dynamics and control using machine learning

et al. 2023

View full text Add to dashboard Cite

Artificial intelligence and mechanical engineering are two mature fields of science that intersect more and more often. Computer-aided mechanical analysis tools, often multibody system software, are very versatile and have revolutionalized many industries. However, as shown by the literature presented in this review, combining the advantages of multibody system dynamics and machine learning creates new and exciting possibilities. For example, the multibody method can assist machine learning by providing synthetic data, while machine learning can provide fast and accurate subsystem models. The intersection of both approaches results in surrogate and hybrid modeling techniques, advanced control algorithms, and optimal design applications. A notable example is the development of autonomous systems for vehicles, robots, and mobile machinery. In our review we have found nontrivial, innovative, and even surprising applications of machine learning and multibody dynamics. This review focuses on applying neural networks, mainly deep learning, in connection with the multibody system method. Over one hundred and fifty papers are covered, and three main research areas are identified and introduced: data-driven modeling, model-based control and estimation, and data-driven control. The paper starts with a primer on machine learning and concludes with future research directions. The main goal is to provide a comprehensive and up-to-date review of existing literature to inspire further research.

show abstract

Reliability evaluation of reinforcement learning methods for mechanical systems with increasing complexity

Manzl,

Rogov,

Gerstmayr

et al. 2023

Multibody Syst Dyn

View full text Add to dashboard Cite

Reinforcement learning (RL) is one of the emerging fields of artificial intelligence (AI) intended for designing agents that take actions in the physical environment. RL has many vital applications, including robotics and autonomous vehicles. The key characteristic of RL is its ability to learn from experience without requiring direct programming or supervision. To learn, an agent interacts with an environment by acting and observing the resulting states and rewards. In most practical applications, an environment is implemented as a virtual system due to cost, time, and safety concerns. Simultaneously, multibody system dynamics (MSD) is a framework for efficiently and systematically developing virtual systems of arbitrary complexity. MSD is commonly used to create virtual models of robots, vehicles, machinery, and humans. The features of RL and MSD make them perfect companions in building sophisticated, automated, and autonomous mechatronic systems. The research demonstrates the use of RL in controlling multibody systems. While AI methods are used to solve some of the most challenging tasks in engineering, their proper understanding and implementation are demanding. Therefore, we introduce and detail three commonly used RL algorithms to control the inverted N-pendulum on the cart. Single-, double-, and triple-pendulum configurations are investigated, showing the capability of RL methods to handle increasingly complex dynamical systems. We show 2D state space zones where the agent succeeds or fails the stabilization. Despite passing randomized tests during training, blind spots may occur where the agent’s policy fails. Results confirm that RL is a versatile, although complex, control engineering approach.

show abstract

Toward Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control

Cited by 44 publications

References 49 publications

Human-Prosthesis Cooperation: Combining Adaptive Prosthesis Control with Visual Feedback Guided Gait

Human-Prosthesis Cooperation: Combining Adaptive Prosthesis Control with Visual Feedback Guided Gait

Multibody dynamics and control using machine learning

Reliability evaluation of reinforcement learning methods for mechanical systems with increasing complexity

Contact Info

Product

Resources

About