Semantic Curiosity for Active Visual Learning

Abstract

In this paper, we study the task of embodied interactive learning for object detection. Given a set of environments (and some labeling budget), our goal is to learn an object detector by having an agent select what data to obtain labels for. How should an exploration policy decide which trajectory should be labeled? One possibility is to use a trained object detector's failure cases as an external reward. However, this will require labeling millions of frames required for training RL policies, which is infeasible. Instead, we explore a self-supervised approach for training our exploration policy by introducing a notion of semantic curiosity. Our semantic curiosity policy is based on a simple observation -the detection outputs should be consistent. Therefore, our semantic curiosity rewards trajectories with inconsistent labeling behavior and encourages the exploration policy to explore such areas. The exploration policy trained via semantic curiosity generalizes to novel scenes and helps train an object detector that outperforms baselines trained with other possible alternatives such as random exploration, prediction-error curiosity, and coverage-maximizing exploration.

ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints

“…These methods often assume access to data from the test environment to start with a viable graph, which may not be available in a new environment. Some works have studied this unseen setting by predicting explorable areas for semantically rich parts of the environment to accelerate visual exploration [31,32]. While these methods can yield promising results in a variety of domains, they come at the cost of high sample complexity (over 10M samples) [23], making them difficult to use in the real world-the most performant algorithms take 10-20 minutes to find goals up to 50m away [33].…”

Learning to Map for Active Semantic Goal Navigation

“…Maximizing epistemic uncertainty is used as a proxy for maximizing information gain [47,41]. We use this uncertainty objective to actively fine-tune our models with a training procedure similar to the active learning approaches leveraged during training presented by [7,13,46]. In particular, [13] uses this active training method to improve performance on a semantic segmentation model, leading us to initially hypothesize analogous results would be possible for our semantic hallucination task.…”

“…We use this uncertainty objective to actively fine-tune our models with a training procedure similar to the active learning approaches leveraged during training presented by [7,13,46]. In particular, [13] uses this active training method to improve performance on a semantic segmentation model, leading us to initially hypothesize analogous results would be possible for our semantic hallucination task. We also use this epistemic uncertainty estimate to construct confidence bounds for our estimated probability distribution which we use to select goals for target-driven navigation at test time.…”

A Survey of Embodied AI: From Simulators to Research Tasks

“…Practical considerations for curiosity have been listed in recent work [63], such as using Proximal Policy Optimization (PPO) for policy optimisation. Curiosity has been used to generate more advanced maps like semantic maps in recent work [66]. Stochasticity poses a serious challenge in the curiosity approach, since the forward-dynamics model can exploit stochasticity [63] for high prediction errors (i.e.…”