Know your limits: Embedding localiser performance models in teach and repeat maps

Churchill, Winston; Tong, Chi Hay; Gurău, Corina; Posner, Ingmar; Newman, Paul

doi:10.1109/icra.2015.7139783

Cited by 16 publications

(14 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accounting for Uncertainty: An AIA that can predict its performance on di erent tasks can provide assurances about competence, predictability, and the situational normality of a given task. Several researchers have worked to improve this ability in visual classi cation [25,51,66,151]. For example, to ensure that visual classi ers don't fail silently in novel scenarios, Zhang et al [151] learned models of errors on training images to predict errors on test images.…”

Section: Commonmentioning

confidence: 99%

“Dave...I can assure you ...that it’s going to be all right ...” A Definition, Case for, and Survey of Algorithmic Assurances in Human-Autonomy Trust Relationships

Israelsen

Ahmed

2019

ACM Comput. Surv.

View full text Add to dashboard Cite

People who design, use, and are a ected by autonomous arti cially intelligent agents want to be able to trust such agents -that is, to know that these agents will perform correctly, to understand the reasoning behind their actions, and to know how to use them appropriately. Many techniques have been devised to assess and in uence human trust in arti cially intelligent agents. However, these approaches are typically ad hoc, and have not been formally related to each other or to formal trust models. is paper presents a survey of algorithmic assurances, i.e. programmed components of agent operation that are expressly designed to calibrate user trust in arti cially intelligent agents. Algorithmic assurances are rst formally de ned and classi ed from the perspective of formally modeled human-arti cially intelligent agent trust relationships. Building on these de nitions, a synthesis of research across communities such as machine learning, human-computer interaction, robotics, e-commerce, and others reveals that assurance algorithms naturally fall along a spectrum in terms of their impact on an agent's core functionality, with seven notable classes ranging from integral assurances (which impact an agent's core functionality) to supplemental assurances (which have no direct e ect on agent performance).Common approaches within each of these classes are identi ed and discussed; bene ts and drawbacks of di erent approaches are also investigated.

show abstract

Section: Commonmentioning

confidence: 99%

“Dave...I can assure you ...that it’s going to be all right ...” A Definition, Case for, and Survey of Algorithmic Assurances in Human-Autonomy Trust Relationships

Israelsen

Ahmed

2019

ACM Comput. Surv.

View full text Add to dashboard Cite

show abstract

“…Similar problems are reported for localisation performance. [10] and [11] propose embedding spatial models of expected localiser performance in localisation maps in order to aid trajectory planners.…”

Section: Related Workmentioning

confidence: 99%

Fit for Purpose? Predicting Perception Performance Based on Past Experience

Gurău

Tong

Posner

2017

Springer Proceedings in Advanced Robotics

Self Cite

View full text Add to dashboard Cite

This paper explores the idea of predicting the likely performance of a robot's perception system based on past experience in the same workspace. In particular, we propose to build a place-specific model of perception performance from observations gathered over time. We evaluate our method in a classical decision making scenario in which the robot must choose when and where to drive autonomously in 60km of driving data from an urban environment. We demonstrate that leveraging visual appearance within a state-of-the-art navigation framework increases the accuracy of our performance predictions.

show abstract

“…A common heuristic in pointbased visual odometry systems is the number of matched features, as these algorithms generally rely on having many correspondences. This heuristic is used in [17] and [18] where the authors report the number of correspondences as a substitute for localization performance. Other works studying visual odometry performance in adverse conditions show correlations between image quantities, e.g.…”

Section: B Heuristic Evaluationmentioning

confidence: 99%

“…Much of this work has been done on vision systems, for instance predicting segmentation or horizon detection failures [25], or traversability estimation failures [26]. Other work has shown prediction of the heuristic performance of a vision-based navigation system [17], [18], and of a classification system [27].…”

Section: Introspectionmentioning

confidence: 99%

Introspective Evaluation of Perception Performance for Parameter Tuning without Ground Truth

Hu¹,

Kantor²

2017

Robotics: Science and Systems XIII

View full text Add to dashboard Cite

Abstract-Modern perception systems are notoriously complex, featuring dozens of interacting parameters that must be tuned to achieve good performance. Conventional tuning approaches require expensive ground truth, while heuristic methods are difficult to generalize. In this work, we propose an introspective ground-truth-free approach to evaluating the performance of a generic perception system. By using the posterior distribution estimate generated by a Bayesian estimator, we show that the expected performance can be estimated efficiently and without ground truth. Our simulated and physical experiments in a demonstrative indoor ground robot state estimation application show that our approach can order parameters similarly to using a ground-truth system, and is able to accurately identify top-performing parameters in varying contexts. In contrast, baseline approaches that reason only about observation log-likelihood fail in the face of challenging perceptual phenomena.

show abstract

Know your limits: Embedding localiser performance models in teach and repeat maps

Cited by 16 publications

References 14 publications

“Dave...I can assure you ...that it’s going to be all right ...” A Definition, Case for, and Survey of Algorithmic Assurances in Human-Autonomy Trust Relationships

“Dave...I can assure you ...that it’s going to be all right ...” A Definition, Case for, and Survey of Algorithmic Assurances in Human-Autonomy Trust Relationships

Fit for Purpose? Predicting Perception Performance Based on Past Experience

Introspective Evaluation of Perception Performance for Parameter Tuning without Ground Truth

Contact Info

Product

Resources

About