Space-indexed dynamic programming

Kolter, J. Zico; Coates, Adam; Ng, Andrew Y.; Gu, Yi; DuHadway, Charles

doi:10.1145/1390156.1390218

Cited by 14 publications

(1 citation statement)

References 14 publications

(11 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Commonly used loss functions are the mean square error or the cross entropy [24]. The Hessian is defined as H ij = ∂ θ i θ j L and has a wide range of applications in ML: it can be used to adapt gradient update to the current loss landscape in the so called 'Newton' method [25], for pruning [26,27] or for interpretability purposes with the influence function [28]. Furthermore, it can also be used to study the local curvature of the loss for a better understanding of the loss landscape and the convergence of NNs.…”

Section: Loss Landscape Of Nns: a Brief Reviewmentioning

confidence: 99%

Characterizing the loss landscape of variational quantum circuits

Huembeli

Dauphin

2021

Quantum Sci. Technol.

View full text Add to dashboard Cite

Machine learning techniques enhanced by noisy intermediate-scale quantum (NISQ) devices and especially variational quantum circuits (VQC) have recently attracted much interest and have already been benchmarked for certain problems. Inspired by classical deep learning, VQCs are trained by gradient descent methods which allow for efficient training over big parameter spaces. For NISQ sized circuits, such methods show good convergence. There are however still many open questions related to the convergence of the loss function and to the trainability of these circuits in situations of vanishing gradients. Furthermore, it is not clear how ‘good’ the minima are in terms of generalization and stability against perturbations of the data and there is, therefore, a need for tools to quantitatively study the convergence of the VQCs. In this work, we introduce a way to compute the Hessian of the loss function of VQCs and show how to characterize the loss landscape with it. The eigenvalues of the Hessian give information on the local curvature and we discuss how this information can be interpreted and compared to classical neural networks. We benchmark our results on several examples, starting with a simple analytic toy model to provide some intuition about the behaviour of the Hessian, then going to bigger circuits, and also train VQCs on data. Finally, we show how the Hessian can be used to adjust the learning rate for faster convergence during the training of variational circuits.

show abstract

Section: Loss Landscape Of Nns: a Brief Reviewmentioning

confidence: 99%

Characterizing the loss landscape of variational quantum circuits

Huembeli

Dauphin

2021

Quantum Sci. Technol.

View full text Add to dashboard Cite

show abstract

Reinforcement Learning in Robotics: A Survey

Kober

Peters

2012

Adaptation, Learning, and Optimization

265

View full text Add to dashboard Cite

Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between modelbased and model-free as well as between value function-based and policy search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research.

show abstract

Distance indexed trajectory generation for a helicopter robot for programming by demonstration

Tanveer

MacDonald

2009

2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics

View full text Add to dashboard Cite

A trajectory generator is presented. While the generator is generic we focus on its application for a programming by demonstration (PbD) system. Using stored task knowledge, our PbD system generates position targets at runtime to guide the robot to achieve specified task goals. This paper addresses the problem of trajectory generation (as a sequence of waypoints) to move from one position target to another.Most existing trajectory generation algorithms produce time indexed trajectories. The traversability of time indexed trajectories can be ensured only if an accurate dynamic model is considered for trajectory generation. However, if the true dynamics are not known, the robot may fail to follow the trajectory. A novel distance indexed trajectory generation method is presented to resolve these issues for a helicopter robot. The approach is evaluated using a helicopter simulator.

show abstract

Space-indexed dynamic programming

Cited by 14 publications

References 14 publications

Characterizing the loss landscape of variational quantum circuits

Characterizing the loss landscape of variational quantum circuits

Reinforcement Learning in Robotics: A Survey

Distance indexed trajectory generation for a helicopter robot for programming by demonstration

Contact Info

Product

Resources

About