Model-Plant Mismatch Compensation Using Reinforcement Learning

Koryakovskiy, Ivan; Kudruss, Manuel; Babuška, Robert; Caarls, Wouter

doi:10.1109/lra.2018.2800106

Cited by 34 publications

(24 citation statements)

References 11 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our framework also provides several new greedy algorithms, including parallel and constrained formulations. We further extend this method to include the effect of control inputs, making it applicable to robotic systems [ 55 ], and we use this procedure to discover Hamiltonians. Finally, our approach provides guidance on model selection, a comprehensive comparison with previous methods, and a careful analysis of noise robustness.…”

Section: Introductionmentioning

confidence: 99%

SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics

2020

View full text Add to dashboard Cite

Accurately modelling the nonlinear dynamics of a system from measurement data is a challenging yet vital topic. The sparse identification of nonlinear dynamics (SINDy) algorithm is one approach to discover dynamical systems models from data. Although extensions have been developed to identify implicit dynamics, or dynamics described by rational functions, these extensions are extremely sensitive to noise. In this work, we develop SINDy-PI (parallel, implicit), a robust variant of the SINDy algorithm to identify implicit dynamics and rational nonlinearities. The SINDy-PI framework includes multiple optimization algorithms and a principled approach to model selection. We demonstrate the ability of this algorithm to learn implicit ordinary and partial differential equations and conservation laws from limited and noisy data. In particular, we show that the proposed approach is several orders of magnitude more noise robust than previous approaches, and may be used to identify a class of ODE and PDE dynamics that were previously unattainable with SINDy, including for the double pendulum dynamics and simplified model for the Belousov–Zhabotinsky (BZ) reaction.

show abstract

Section: Introductionmentioning

confidence: 99%

SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics

2020

View full text Add to dashboard Cite

show abstract

“…Our work toward compensating for modeling error with datadriven learning is similar to Sun et al (2019) where authors use deep learning to predict physics-based modeling error of water resources, Kaheman et al (2019) where they present an algorithm to learn a discrepancy model on an double inverted pendulum, and Della Santina et al (2020) where the authors augment a model-based disturbance observer with a learned correction factor on a soft robot. Most similar to our work is that of Koryakovskiy et al (2018) where they augment a non-linear model predictive controller with various forms of learned actions to compensate for model-plant mismatch on a rigid humanoid robot. Other works that include using neural networks as the backbone for predictive control are Piche et al (2000) and Lu and Tsai (2008).…”

Section: Related Workmentioning

confidence: 96%

Using First Principles for Deep Learning and Model-Based Control of Soft Robots

et al. 2021

View full text Add to dashboard Cite

Model-based optimal control of soft robots may enable compliant, underdamped platforms to operate in a repeatable fashion and effectively accomplish tasks that are otherwise impossible for soft robots. Unfortunately, developing accurate analytical dynamic models for soft robots is time-consuming, difficult, and error-prone. Deep learning presents an alternative modeling approach that only requires a time history of system inputs and system states, which can be easily measured or estimated. However, fully relying on empirical or learned models involves collecting large amounts of representative data from a soft robot in order to model the complex state space–a task which may not be feasible in many situations. Furthermore, the exclusive use of empirical models for model-based control can be dangerous if the model does not generalize well. To address these challenges, we propose a hybrid modeling approach that combines machine learning methods with an existing first-principles model in order to improve overall performance for a sampling-based non-linear model predictive controller. We validate this approach on a soft robot platform and demonstrate that performance improves by 52% on average when employing the combined model.

show abstract

“…Koryakovskiy et al [23] argued that learning from scratch approaches are not applicable on real robots due to involving many trials with exploratory actions. To circumvent this problem, they combined RL with a model-based control using two different approaches to compensate model-plant mismatches.…”

Section: B Combination Of Model-based and MLmentioning

confidence: 99%

A CPG-Based Agile and Versatile Locomotion Framework Using Proximal Symmetry Loss

Kasaei¹,

Abreu²,

Lau³

et al. 2021

Preprint

View full text Add to dashboard Cite

Humanoid robots are made to resemble humans but their locomotion abilities are far from ours in terms of agility and versatility. When humans walk on complex terrains, or face external disturbances, they combine a set of strategies, unconsciously and efficiently, to regain stability. This paper tackles the problem of developing a robust omnidirectional walking framework, which is able to generate versatile and agile locomotion on complex terrains. The Linear Inverted Pendulum Model and Central Pattern Generator concepts are used to develop a closed-loop walk engine that is combined with a reinforcement learning module. This module learns to regulate the walk engine parameters adaptively and generates residuals to adjust the robot's target joint positions (residual physics). Additionally, we propose a proximal symmetry loss to increase the sample efficiency of the Proximal Policy Optimization algorithm by leveraging model symmetries. The effectiveness of the proposed framework was demonstrated and evaluated across a set of challenging simulation scenarios. The robot was able to generalize what it learned in one scenario, by displaying humanlike locomotion skills in unforeseen circumstances, even in the presence of noise and external pushes.

show abstract

Model-Plant Mismatch Compensation Using Reinforcement Learning

Cited by 34 publications

References 11 publications

SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics

SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics

Using First Principles for Deep Learning and Model-Based Control of Soft Robots

A CPG-Based Agile and Versatile Locomotion Framework Using Proximal Symmetry Loss

Contact Info

Product

Resources

About