Koopman Operator–Based Knowledge-Guided Reinforcement Learning for Safe Human–Robot Interaction

Sinha, Anirban; Wang, Yue

doi:10.3389/frobt.2022.779194

Cited by 6 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the subsequent section, we draw comparisons with two methods based on the Decision Transformer: the original Decision Transformer (DT) [11] and the Q-learning Decision Transformer (QDT) [57]. Additionally, we include a behavior cloning-based method (TS+BC) [19], as well as two offline Q-learning methods, namely S4RL [45] and IQL [26], in our comparisons.…”

Section: Baseline Methodsmentioning

confidence: 99%

Multi-focus image fusion: Transformer and shallow feature attention matters

Jiang

Hua

et al. 2023

Displays

View full text Add to dashboard Cite

Section: Baseline Methodsmentioning

confidence: 99%

Multi-focus image fusion: Transformer and shallow feature attention matters

Jiang

Hua

et al. 2023

Displays

View full text Add to dashboard Cite

“…Such techniques can be applied to state representations to generate unseen data points. In [31,29], different data augmentation schemes are compared and applied to off-the-shelf RL algorithms. However, naively applying data augmentation to DL/RL can cause new problems.…”

Section: Incomplete Datamentioning

confidence: 99%

A Survey of Demonstration Learning

Correia¹,

Alexandre²

2023

Preprint

View full text Add to dashboard Cite

With the fast improvement of machine learning, reinforcement learning (RL) has been used to automate human tasks in different areas. However, training such agents is difficult and restricted to expert users. Moreover, it is mostly limited to simulation environments due to the high cost and safety concerns of interactions in the real world. Demonstration Learning is a paradigm in which an agent learns to perform a task by imitating the behavior of an expert shown in demonstrations. It is a relatively recent area in machine learning, but it is gaining significant traction due to having tremendous potential for learning complex behaviors from demonstrations. Learning from demonstration accelerates the learning process by improving sample efficiency, while also reducing the effort of the programmer. Due to learning without interacting with the environment, demonstration learning would allow the automation of a wide range of real world applications such as robotics and healthcare. This paper provides a survey of demonstration learning, where we formally introduce the demonstration problem along with its main challenges and provide a comprehensive overview of the process of learning from demonstrations from the creation of the demonstration data set, to learning methods from demonstrations, and optimization by combining demonstration learning with different machine learning methods. We also review the existing benchmarks and identify their strengths and limitations. Additionally, we discuss the advantages and disadvantages of the paradigm as well as its main applications. Lastly, we discuss our perspective on open problems and research directions for this rapidly growing field.

show abstract

“…Similarly, end-effector motion of industrial robotic arms around humans requires environmental state prediction for safe path planning. This was done in one study where the Koopman operator was directly solved for by taking the pseudo-inverse involved in minimization of the Frobenius norm (a computationally expensive operation) [33]. The same objective of safe path planning was also achieved in [12] with the use of the adjoint Koopman operator and in [94] using the Stochastic Koopman Operator.…”

Section: Human-robot Collaborationmentioning

confidence: 99%

Vehicular Applications of Koopman Operator Theory—A Survey

2023

View full text Add to dashboard Cite

Koopman operator theory has proven to be a promising approach to nonlinear system identification and global linearization. For nearly a century, there had been no efficient means of calculating the Koopman operator for applied engineering purposes. The introduction of a recent computationally efficient method in the context of fluid dynamics, which is based on the system dynamics decomposition to a set of normal modes in descending order, has overcome this long-lasting computational obstacle. The purely data-driven nature of Koopman operators holds the promise of capturing unknown and complex dynamics for reduced-order model generation and system identification, through which the rich machinery of linear control techniques can be utilized. Given the ongoing development of this research area and the many existing open problems in the fields of smart mobility and vehicle engineering, a survey of techniques and open challenges of applying Koopman operator theory to this vibrant area is warranted. This review focuses on the various solutions of the Koopman operator which have emerged in recent years, particularly those focusing on mobility applications, ranging from characterization and component-level control operations to vehicle performance and fleet management. Moreover, this comprehensive review of over 100 research papers highlights the breadth of ways Koopman operator theory has been applied to various vehicular applications with a detailed categorization of the applied Koopman operator-based algorithm type. Furthermore, this review paper discusses theoretical aspects of Koopman operator theory that have been largely neglected by the smart mobility and vehicle engineering community and yet have large potential for contributing to solving open problems in these areas.

show abstract

Koopman Operator–Based Knowledge-Guided Reinforcement Learning for Safe Human–Robot Interaction

Cited by 6 publications

References 19 publications

Multi-focus image fusion: Transformer and shallow feature attention matters

Multi-focus image fusion: Transformer and shallow feature attention matters

A Survey of Demonstration Learning

Vehicular Applications of Koopman Operator Theory—A Survey

Contact Info

Product

Resources

About