Combining Reinforcement Learning and Belief Revision - A Learning System for Active Vision

Abstract

Computer vision can highly benefit from modern learning methods. In the context of an active vision environment we introduce a machine learning approach which is able to learn strategies of object acquisition. We propose a hybrid learning method, called Sphinx, that combines two approaches originating from seperate disciplines of computer science, namely reinforcement learning on the one hand and belief revision on the other. The former represents knowledge in a numerical way, while the latter is based on symbolic logic and allows reasoning. Sphinx is designed according to human cognition and interacts with its environment by rotating objects depending on past perceptions to acquire those views which are advantageous for recognition. Our method was successfully applied in simulations of object categorization tasks.

Visualization of processes in self-learning systems

“…We restrict ourselves to describing only those aspects of the system, which are relevant for this purpose. For details refer to [8]. The iterative two-level architecture of Sphinx is displayed in figure 1.…”

“…Only recently these fields of machine learning slowly begin to merge, taking advantage of the mutual benefits of both concepts. One such approach that combines both ideas is the Sphinx system described in [8], [9]. We take this system as an example and a starting point to develop our ideas on the visualization of learning processes.…”

“…The symbolic level of if-then rules is realized by means of belief revision. Before the execution of the very first learning step neither any symbolic knowledge nor any numeric knowledge has been acquired, as detailed in[8].of a specific state is the conjunction of the corresponding literals of all of these variables.The possible actions are two rotation actions (to the left and to the right), one recognition action for each of 9 objects • Recognize = {Ball, Bird, Bottle, DVD, Football, House, Pott, TetraPack, Tree} and the decision to not recognize anything. Thus, here we have one variable with 12 possible values:…”

Improved Revision of Ranking Functions for the Generalization of Belief in the Context of Unobserved Variables

“…In this case, the meaning of S ⇒ A and (A|S) is the same. Therefore, on a first attempt, we use (κ * (A|S)) to revise κ with a conditional analogous to (Leopold et al, 2008). Then, we will examine the consequences of such a decision.…”

“…While humans are able to learn top-down or bottom-up (Sun et al, 2006), we will focus on the bottom-up part only. A combination of RL and BR has been proposed before (Leopold et al, 2008), influenced by (Sun et al, 2001) and (Ye et al, 2003). While we have already described the general idea of our approach in (Häming and Peters, 2010), we present here the detailed formalism and give a theoretical justification.…”