Uncertainty-Aware Data Aggregation for Deep Imitation Learning

Abstract

Estimating statistical uncertainties allows autonomous agents to communicate their confidence during task execution and is important for applications in safety-critical domains such as autonomous driving. In this work, we present the uncertainty-aware imitation learning (UAIL) algorithm for improving end-to-end control systems via data aggregation. UAIL applies Monte Carlo Dropout to estimate uncertainty in the control output of end-to-end systems, using states where it is uncertain to selectively acquire new training data. In contrast to prior data aggregation algorithms that force human experts to visit sub-optimal states at random, UAIL can anticipate its own mistakes and switch control to the expert in order to prevent visiting a series of sub-optimal states. Our experimental results from simulated driving tasks demonstrate that our proposed uncertainty estimation method can be leveraged to reliably predict infractions. Our analysis shows that UAIL outperforms existing data aggregation algorithms on a series of benchmark tasks.

Intelligent control of flexible joint based on cooperative learning theory

“…It is desired that the actual system output will achieve the response of a typical second-order system at a damping ratio 𝜉 = 0.707, the reward function is defined as follows: where 𝑠 (𝑘) denotes the 𝑘𝑡ℎ tracking error of the actual system, 𝑒(𝑘) denotes the 𝑘𝑡ℎ data of the ideal dataset 𝐸𝑟, and 𝜌 serves to adjust the convergence rate of the algorithm. From equation (10), it can be seen that when the actual position tracking error is closer to the ideal tracking error, the reward value 𝑟 will keep approaching 1, otherwise the reward value 𝑟 is close to 0. With the RL method, coach is trained and network weight is updated.…”

“…Second, as traditional training methods for machine learning-based intelligent controllers are generally used separately, different methods need to be reselected for different usage scenarios. The effectiveness of imitation learning [9,10] with dataset limitations depends on the size and features of the dataset, and models trained based on imitation learning have a certain confidence level in prediction and may not make the best decisions. The complex relationship between the cost function or reward function and the optimal decision needs to be determined in reinforcement learning [11], and the ideal cost function is difficult to implement in practice.…”

Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworks

“…• Uncertainty-Aware Data Aggregation for Deep Imitation Learning (UAIL) [39]: UAIL gathers training data by estimating the output's uncertainty at a sub-optimal state. Monte Carlo Dropout is used for uncertainty estimation, in which the output distribution is computed using multiple dropout masks at each level, then the statistics of this distribution are used to calculate the uncertainty score, which is then compared to the uncertainty threshold.…”

“…Effective for less complex DNN Fuzzy neural network Trajectory tracking under varied dynamics [39] The online learning of a pretrained deep fuzzy neural network-based controller improves the control of nonlinear system under diverse and varied operating conditions (i.e different payloads, height, and speed).…”

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

“…In their approach, they gathered human demonstrations for grasping the sheet and failure detection, by utilizing pre-trained YOLO features in order to facilitate the learning of deep neural network policies. Other works on the execution of folding cloths can be found in [199]- [203]. Instead of improving the synthetic objects to be indistinguishable from real objects, Abolghasemi and Bölöni [204] have trained the vision system to accept synthetic objects as real.…”