Predictive information and explorative behavior of autonomous robots

Abstract

Abstract.Measures of complexity are of immediate interest for the field of autonomous robots both as a means to classify the behavior and as an objective function for the autonomous development of robot behavior. In the present paper we consider predictive information in sensor space as a measure for the behavioral complexity of a two-wheel embodied robot moving in a rectangular arena with several obstacles. The mutual information (MI) between past and future sensor values is found empirically to have a maximum for a behavior which is both explorative and sensitive to the environment. This makes predictive information a prospective candidate as an objective function for the autonomous development of such behaviors. We derive theoretical expressions for the MI in order to obtain an explicit update rule for the gradient ascent dynamics. Interestingly, in the case of a linear or linearized model of the sensorimotor dynamics the structure of the learning rule derived depends only on the dynamical properties while the value of the MI influences only the learning rate. In this way the problem of the prohibitively large sampling times for information theoretic measures can be circumvented. This result can be generalized and may help to derive explicit learning rules from complexity theoretic measures. PACS

The Intrinsic Cause-Effect Power of Discrete Dynamical Systems—From Elementary Cellular Automata to Adapting Animats

“…In addition to the state entropy, in [19] we also assessed how the sensory-motor mutual information (ISMMI) [34] and predictive information (IPred) [35] of the animats as defined in [19,36] evolved during adaptation. ISMMI measures the differentiation of the observed input-output behavior of the animats' sensors and motors.…”

A Novel Information Measure for Predictive Learning in a Social System Setting

“…While Polani [11,9] and Lungarella [16,10] used the empowerment measure as a general cost function to optimise the agent's behaviour or evolution, we use it as the upper bound of the MI to measure the efficiency of the sensory-motor loop use. Ay in his work [2] uses an adaptive controller which maximises the excess entropy (the mutual information between past and present) at the input side to achieve a working regime exploratory and sensitive to the environment. We can calculate the MI for this case by considering the reflex as the present input and the predictor as the past history.…”

Behavioral analysis of differential hebbian learning in closed-loop systems

“…Also, different from our attempt, these authors analyzed the system in a reflex-based closed-loop scenario where no learning had been applied. Ay et al (2008) and Der et al (2008) used a predictive information measure (PI, mutual information between past and future sensor values) to evaluate behavioral complexity of agents and to use PI as an objective function for the agents' adaptation, however, similar to Lungarella et al (2005), only looking at the input space.…”