An Ergodic Measure for Active Learning From Equilibrium

Abraham, Ian; Prabhakar, Ahalya; Murphey, Todd D.

doi:10.1109/tase.2020.3043636

Cited by 21 publications

(19 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…( 3)) has linear time complexity, allowing us to avoid the computational complexity of IRL. As documented in prior work, ergodic control scales well to relatively high-dimensional spaces-it has been implemented on the half cheetah example with 26 observables and continuous actions [27].…”

Section: Related Workmentioning

confidence: 97%

Ergodic imitation: Learning from what to do and what not to do

Kalinowska

Prabhakar

Fitzsimons

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

With growing access to versatile robotics, it is beneficial for end users to be able to teach robots tasks without needing to code a control policy. One possibility is to teach the robot through successful task executions. However, nearoptimal demonstrations of a task can be difficult to provide and even successful demonstrations can fail to capture task aspects key to robust skill replication. Here, we propose a learning from demonstration (LfD) approach that enables learning of robust task definitions without the need for near-optimal demonstrations. We present a novel algorithmic framework for learning tasks based on the ergodic metric-a measure of information content in motion. Moreover, we make use of negative demonstrations-demonstrations of what not to do-and show that they can help compensate for imperfect demonstrations, reduce the number of demonstrations needed, and highlight crucial task elements improving robot performance. In a proofof-concept example of cart-pole inversion, we show that negative demonstrations alone can be sufficient to successfully learn and recreate a skill. Through a human subject study with 24 participants, we show that consistently more information about a task can be captured from combined positive and negative (posneg) demonstrations than from the same amount of just positive demonstrations. Finally, we demonstrate our learning approach on simulated tasks of target reaching and table cleaning with a 7-DoF Franka arm. Our results point towards a future with robust, data-efficient LfD for novice users.

show abstract

Section: Related Workmentioning

confidence: 97%

Ergodic imitation: Learning from what to do and what not to do

Kalinowska

Prabhakar

Fitzsimons

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is due to the computational complexity and storage demanded in working with the ergodic metric and the control policy derived from it. Several authors deviated from the original ergodic control formulation to tackle this limitation, see e.g., [9], [10]. In [9] and [10], the authors relied on a different ergodic metric based on a Kullback-Leibler (KL) divergence measure for finite sensor footprint, where the control policy is obtained using sampling-based techniques.…”

Section: A Ergodic Controlmentioning

confidence: 99%

“…Several authors deviated from the original ergodic control formulation to tackle this limitation, see e.g., [9], [10]. In [9] and [10], the authors relied on a different ergodic metric based on a Kullback-Leibler (KL) divergence measure for finite sensor footprint, where the control policy is obtained using sampling-based techniques. Samplingbased methods avoid the curse of dimensionality but they are still computationally expensive to address the real-time computational requirements of robotics systems.…”

Section: A Ergodic Controlmentioning

confidence: 99%

“…As explained above, the existence of low-rank structure (i.e., a low maximal TT-rank) of a given tensor is closely related to the separability of the underlying multivariate function. Although separability of functions is not a very well understood problem, it is known that smoothness 10 and symmetry of functions often induces better 11 separability of the functions. This has been exploited to solve PDEs and problems involving high-dimensional integration [26], and it will be used in this paper to solve the challenges in ergodic control.…”

Section: Tensor Train Decompositionmentioning

confidence: 99%

“…which, analogously to (10), maps from an arbitrary tangent space T g S 3 back to the manifold through…”

Section: B Ergodic Control On Riemannian Manifoldsmentioning

confidence: 99%

See 2 more Smart Citations

Ergodic Exploration using Tensor Train: Applications in Insertion Tasks

Shetty¹,

Silvério²,

Calinon³

2021

Preprint

View full text Add to dashboard Cite

By generating control policies that create natural search behaviors in autonomous systems, ergodic control provides a principled solution to address tasks that require exploration. A large class of ergodic control algorithms relies on spectral analysis, which suffers from the curse of dimensionality, both in storage and computation. This drawback has prohibited the application of ergodic control in robot manipulation since it often requires exploration in state space with more than 2 dimensions. Indeed, the original ergodic control formulation will typically not allow exploratory behaviors to be generated for a complete 6D end-effector pose. In this paper, we propose a solution for ergodic exploration based on spectral analysis in multidimensional spaces using low-rank tensor approximation techniques. We rely on tensor train decomposition, a recent approach from multilinear algebra for low-rank approximation and efficient computation of multidimensional arrays. The proposed solution is efficient both computationally and storage-wise, hence making it suitable for its online implementation in robotic systems. The approach is applied to a peg-in-hole insertion task using a 7-axis Franka Emika Panda robot, where ergodic exploration allows the task to be achieved without requiring the use of force/torque sensors.

show abstract