Knowing when we don't know: Introspective classification for mission-critical decision making

Grimmett, Hugo; Paul, Rohan; Triebel, Rudolph; Posner, Ingmar

doi:10.1109/icra.2013.6631221

Cited by 31 publications

(25 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Concretely, we introduce a formal definition of over-and underconfidence of a classifier on a given data set. This is related to the intuitive notion of introspection introduced by Grimmett et al [1], however with the di↵erence that our formulation can explicitly quantify the inherent trade-o↵ between a high number of detected misclassifications and a high number of correct classifications that are not further used for training.…”

Section: A Related Workmentioning

confidence: 99%

“…If we follow the first idea, then a good choice for a classifier is the Gaussian Process classifier (GPC), as was shown earlier [1], [15], because due to its capability to marginalize over a range of potential models, its uncertainty estimates are more reliable because they correlate more with actual misclassifications (see [1] for more details). One major problem however, is its huge demand in run time and memory.…”

Section: Online Confidence Boostingmentioning

confidence: 99%

“…In [1], [15] it was shown that a GPC tends to be much less overconfident than other classifiers such as a Support Vector Machine (SVM) or LogitBoost. However, here we show that Confidence Boosting mitigates this issue at least for boosting methods, and we relate that not only to the fact that the confidence is used to compute the sample weights, but also to the choice of the weak classifiers, namely random forests.…”

Section: Comparison To the Gpcmentioning

confidence: 99%

“…In the case of object classification this means that the required amount of human-labeled training data should be small. Finally, for a mobile robot it is very advantageous to have a reliable measure of confidence along with the classification results, because often important and safety-critical decisions depend on them, and a restriction to very confident classifications can help to avoid accidents, for example in autonomous driving applications [1], [2].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Active online confidence boosting for efficient object classification

Mund

Triebel

Cremers

2015

2015 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Abstract-We present a novel e cient algorithm for object classification. Our method is based on the active learning framework, in which training and classification are performed in loops, and new ground truth labels are queried from the supervisor in each loop. Our underlying classifier is from the family of boosting methods, but in contrast to earlier methods, our Confidence Boosting particularly focusses on misclassified samples that have a high classification confidence associated. We show that weighting these samples more than others leads to a decrease of overconfidence, for which we give a formal definition. As a result, our classifier is better suited for active learning, leading to steeper learning curves and less required label queries. We show the benefits of our approach on standard data sets from machine learning and robotics.

show abstract

Section: A Related Workmentioning

confidence: 99%

Section: Online Confidence Boostingmentioning

confidence: 99%

Section: Comparison To the Gpcmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Active online confidence boosting for efficient object classification

Mund

Triebel

Cremers

2015

2015 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

show abstract

“…To combat this and also maximise flexibility, we use activelearning algorithms in which a human works in the loop with the machine to answer the most challenging classifications as chosen by the algorithm, thus reducing the labelling quantity for a given desired performance level. In order to further reduce the labelling effort, we make use of introspective active learning algorithms [14].…”

Section: Localisation and Multi-session Mappingmentioning

confidence: 99%

Integrating metric and semantic maps for vision-only automated parking

Grimmett

Buerki

Paz

et al. 2015

2015 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

We present a framework for integrating two layers of map which are often required for fully automated operation: metric and semantic. Metric maps are likely to improve with subsequent visitations to the same place, while semantic maps can comprise both permanent and fluctuating features of the environment. However, it is not clear from the state of the art how to update the semantic layer as the metric map evolves.The strengths of our method are threefold: the framework allows for the unsupervised evolution of both maps as the environment is revisited by the robot; it uses vision-only sensors, making it appropriate for production cars; and the human labelling effort is minimised as far as possible while maintaining high fidelity. We evaluate this on two different car parks with a fully automated car, performing repeated automated parking manoeuvres to demonstrate the robustness of the system.

show abstract