Belief tree search for active object recognition

Abstract

Active Object Recognition (AOR) has been approached as an unsupervised learning problem, in which optimal trajectories for object inspection are not known and are to be discovered by reducing label uncertainty measures or training with reinforcement learning. Such approaches have no guarantees of the quality of their solution. In this paper, we treat AOR as a Partially Observable Markov Decision Process (POMDP) and find near-optimal policies on training data using Belief Tree Search (BTS) on the corresponding belief Markov Decision Process (MDP). AOR then reduces to the problem of knowledge transfer from near-optimal policies on training set to the test set. We train a Long Short Term Memory (LSTM) network to predict the best next action on the training set rollouts. We sho that the proposed AOR method generalizes well to novel views of familiar objects and also to novel objects. We compare this supervised scheme against guided policy search, and find that the LSTM network reaches higher recognition accuracy compared to the guided policy method. We further look into optimizing the observation function to increase the total collected reward of optimal policy. In AOR, the observation function is known only approximately. We propose a gradient-based method update to this approximate observation function to increase the total reward of any policy. We show that by optimizing the observation function and retraining the supervised LSTM network, the AOR performance on the test set improves significantly.

Continuous Viewpoint Planning in Conjunction with Dynamic Exploration for Active Object Recognition

“…In this subsection, several baselines [ 10 ] and state-of-the-art VP approaches [ 11 , 12 , 16 ] are employed for experiment comparison with our continuous VP method, which are showed as follows: Random policy [ 10 ] plans a random action from the action space with uniform probability; Sequential policy [ 10 ] moves the agent to the next immediate position in the same direction; DQL policy [ 11 , 12 ] exploits deep Q-Learning algorithm to learn a discrete VP policy. The discrete action space is ; E-TRPO policy [ 16 ] develops a continuous VP method which is implemented by trust region policy optimization [ 17 ] and extreme learning machine [ 18 ].…”

“…DQL policy [ 11 , 12 ] exploits deep Q-Learning algorithm to learn a discrete VP policy. The discrete action space is ;…”

“…Similarly, Liu et al develop a hierarchical local-receptive-field-based extreme learning machine architecture to learn the state representation and utilize Q-learning to find the optimal policy [ 10 ]. In [ 11 ], researchers treat AOR as a Partially Observable Markov Decision Process and find corresponding action-values of training data using belief tree search. All above methods explore discrete viewpoint space, which may miss a few important object information owing to the quantization error of viewpoint.…”

“…A good VP policy can greatly ameliorate the recognition performance. In recent years, reinforcement learning has attracted growing research attention on viewpoint planning [ 8 , 9 , 10 , 11 , 12 ]. The agent is able to learn a good VP policy under the guidance of hand-designed reward functions.…”

“…The agent is able to learn a good VP policy under the guidance of hand-designed reward functions. The main algorithms involved in the learning process are dynamic programming [ 8 ] and Q-Learning [ 9 , 10 , 11 , 12 ]. Both dynamic programming based and Q-Learning based methods have made a great contribution to AOR.…”