An Information-Theoretic Approach for the Quantification of Relevance

Polani, Daniel; Martinetz, Thomas; Kim, Jan T.

doi:10.1007/3-540-44811-x_82

Cited by 25 publications

(25 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To do so, one interprets an agents' actions as the relevance indicator variables of the bottleneck formalism. This provides an informational treatment of single-decision sequential (Markovian) decision problems (Polani et al 2001(Polani et al , 2006, quantifying how much informational processing power is needed to realize a policy that achieves a particular value/reward level. Since we determine the relevance of information by the value it allows the agent to achieve, we speak in this context also of valuable information 2 .…”

Section: Coupling Information With Decisions and Actionsmentioning

confidence: 99%

“…Let us emphasize again that the philosophy of the present approach is intimately related to the concept of relevant information (Tishby et al 1999;Polani et al 2001Polani et al , 2006 which quantifies the minimal informational cost for individual decisions an agent needs to take to achieve a given future expected cumulated reward (i.e. value).…”

Section: Actions As Codingmentioning

confidence: 99%

“…We consider complete decision sequences and compute their corresponding information-to-go during the whole course of a sequence (as opposed to the information processed in each single decision, as in Polani et al 2001Polani et al , 2006. We will combine the Bayesian Network formalism with the MDP picture to derive tradeoffs between the reward achieved in the MDP and the informational effort or cost required to achieve this reward.…”

Section: Bellman Recursion For Sequential Information Processingmentioning

confidence: 99%

See 2 more Smart Citations

Information Theory of Decisions and Actions

Tishby

Polani²

2010

Perception-Action Cycle

Self Cite

219

262

View full text Add to dashboard Cite

The perception-action cycle is often defined as "the circular flow of information between an organism and its environment in the course of a sensory guided sequence of actions towards a goal" (Fuster 2001(Fuster , 2006. The question we address in this paper is in what sense this "flow of information" can be described by Shannon's measures of information introduced in his mathematical theory of communication. We provide an affirmative answer to this question using an intriguing analogy between Shannon's classical model of communication and the Perception-Action-Cycle. In particular, decision and action sequences turn out to be directly analogous to codes in communication, and their complexity -the minimal number of (binary) decisions required for reaching a goal -directly bounded by information measures, as in communication. This analogy allows us to extend the standard Reinforcement Learning framework. The latter considers the future expected reward in the course of a behaviour sequence towards a goal (value-to-go). Here, we additionally incorporate a measure of information associated with this sequence: the cumulated information processing cost or bandwidth required to specify the future decision and action sequence (information-to-go).Using a graphical model, we derive a recursive Bellman optimality equation for information measures, in analogy to Reinforcement Learning; from this, we obtain new algorithms for calculating the optimal trade-off between the value-to-go and the required information-to-go, unifying the ideas behind the Bellman and the Blahut-Arimoto iterations. This trade-off between value-to-go and information-togo provides a complete analogy with the compression-distortion trade-off in source coding. The present new formulation connects seemingly unrelated optimization problems. The algorithm is demonstrated on grid world examples.

show abstract

Section: Coupling Information With Decisions and Actionsmentioning

confidence: 99%

Section: Actions As Codingmentioning

confidence: 99%

Section: Bellman Recursion For Sequential Information Processingmentioning

confidence: 99%

See 1 more Smart Citation

Information Theory of Decisions and Actions

Tishby

Polani²

2010

Perception-Action Cycle

Self Cite

219

262

View full text Add to dashboard Cite

show abstract

“…A discussion of the meaning of sensor data in [9] also suggests that the meaning of something is grounded in how it is used by the entity interpreting it.…”

Section: Meaning For Sensor Datamentioning

confidence: 99%

Self-Adapting Context Definition

O’Connor

Cunningham

Cahill

2007

First International Conference on Self-Adaptive and Self-Organizing Systems (SASO 2007)

View full text Add to dashboard Cite

Many approaches to context-aware computing have focused on providing means for developers to define application contexts. In these approaches, correct application behavior depends on the developer defining the right contexts. The application cannot adapt incorrectly defined contexts as they are statically defined by the developer. We propose a method by which an application can use feedback to evaluate its contexts and adapt them where necessary. This results in more accurate context definitions and should lead to improved application performance. We discuss how this is achieved using Q-learning, and present a scenario and experimental results to support our approach.

show abstract

“…[4] introduces an informationtheoretic view of perception and actuation and discusses usefulness to an agent as a means to attribute meaning to information. This is further formalized in the context of relevant information in [5]. Relevant information "flows" from sensors to actuators, thus connecting them.…”

Section: Introductionmentioning

confidence: 99%