The principles of goal-directed decision-making: from neural mechanisms to computation and robotics

Pezzulo, Giovanni; Verschure, Paul F. M. J.; Balkenius, Christian; Pennartz, Cyriel M. A.

doi:10.1098/rstb.2013.0470

Cited by 25 publications

(13 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…satiation) or a desired temperature for a thermostat. The ontological status of these set points is innocuous with respect to issues of 'evolutionary thinking versus finalisms', and it is an empirical question whether animals use internally represented goal states with various levels of complexity (from 'my finger pressing a button' to 'my face on the cover of Time magazine') to guide their decisions [29,30], as opposed to simpler (e.g. stimulus-response mechanisms)-or both.…”

Section: Top-down Models: a Complement To Emergencementioning

confidence: 99%

Top-down models in biology: explanation and control of complex living systems above the molecular level

Pezzulo

Levin

2016

J. R. Soc. Interface.

Self Cite

172

180

View full text Add to dashboard Cite

It is widely assumed in developmental biology and bioengineering that optimal understanding and control of complex living systems follows from models of molecular events. The success of reductionism has overshadowed attempts at top-down models and control policies in biological systems. However, other fields, including physics, engineering and neuroscience, have successfully used the explanations and models at higher levels of organization, including least-action principles in physics and control-theoretic models in computational neuroscience. Exploiting the dynamic regulation of pattern formation in embryogenesis and regeneration requires new approaches to understand how cells cooperate towards large-scale anatomical goal states. Here, we argue that top-down models of pattern homeostasis serve as proof of principle for extending the current paradigm beyond emergence and molecule-level rules. We define top-down control in a biological context, discuss the examples of how cognitive neuroscience and physics exploit these strategies, and illustrate areas in which they may offer significant advantages as complements to the mainstream paradigm. By targeting system controls at multiple levels of organization and demystifying goal-directed (cybernetic) processes, top-down strategies represent a roadmap for using the deep insights of other fields for transformative advances in regenerative medicine and systems bioengineering.

show abstract

Section: Top-down Models: a Complement To Emergencementioning

confidence: 99%

Top-down models in biology: explanation and control of complex living systems above the molecular level

Pezzulo

Levin

2016

J. R. Soc. Interface.

Self Cite

172

180

View full text Add to dashboard Cite

show abstract

“…At the highest level of the decision making and control hierarchy, human reward systems reflect changing goals and subgoals, and we are only beginning to understand how goals are actually coded in the brain, how we switch between goals, and how the cost functions used in learning depend on goal state (O'Reilly et al, 2014b;Buschman and Miller, 2014;Pezzulo et al, 2014). Goal hierarchies are beginning to be incorporated into deep learning .…”

Section: Hierarchical Controlmentioning

confidence: 99%

Towards an integration of deep learning and neuroscience

Marblestone

Wayne

Körding

2016

Preprint

View full text Add to dashboard Cite

Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial neural networks tend to eschew precisely designed codes, dynamics or circuits in favor of brute force optimization of a cost function, often using simple and relatively uniform initial architectures. Two recent developments have emerged within machine learning that create an opportunity to connect these seemingly divergent perspectives. First, structured architectures are used, including dedicated systems for attention, recursion and various forms of short-and long-term memory storage. Second, cost functions and training procedures have become more complex and are varied across layers and over time. Here we think about the brain in terms of these ideas. We hypothesize that (1) the brain optimizes cost functions, (2) these cost functions are diverse and differ across brain locations and over development, and (3) optimization operates within a pre-structured architecture matched to the computational problems posed by behavior. Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism. We suggest directions by which neuroscience could seek to refine and test these hypotheses.

show abstract

“…In human psychology and cognitive science, decision-making is usually thought to depend on explicit thinking and deliberation by the individual (Baars and Gage, 2010). But even in humans, much of decision-making is served by model-based and goaldirected mechanisms that are not necessarily linked with consciousness (Botvinick and Cohen, 2014;Pezzulo et al, 2014;Bach and Dayan, 2017). The focus in these disciplines is on general and universal proximate mechanisms (e.g., Bayesian analysis or a set of broadly applicable heuristic rules, e.g., tallying) to account for rational decision-making across various domains of situations.…”

Section: Perspectives On Decision-makingmentioning

confidence: 99%

“…is used in neurobiology (Gold and Shadlen, 2007;Cisek and Kalaska, 2010;Brody and Hanks, 2016). A focus on the general architecture for optimal and resilient action selection is common in robotics and artificial intelligence (Arkin, 1998;Pezzulo et al, 2014;Lewis and Canamero, 2016).…”

Section: Perspectives On Decision-makingmentioning

confidence: 99%

Decision-Making From the Animal Perspective: Bridging Ecology and Subjective Cognition

et al. 2019

View full text Add to dashboard Cite

Organisms have evolved to trade priorities across various needs, such as growth, survival, and reproduction. In naturally complex environments this incurs high computational costs. Models exist for several types of decisions, e.g., optimal foraging or life history theory. However, most models ignore proximate complexities and infer simple rules specific to each context. They try to deduce what the organism must do, but do not provide a mechanistic explanation of how it implements decisions. We posit that the underlying cognitive machinery cannot be ignored. From the point of view of the animal, the fundamental problems are what are the best contexts to choose and which stimuli require a response to achieve a specific goal (e.g., homeostasis, survival, reproduction). This requires a cognitive machinery enabling the organism to make predictions about the future and behave autonomously. Our simulation framework includes three essential aspects: (a) the focus on the autonomous individual, (b) the need to limit and integrate information from the environment, and (c) the importance of goal-directed rather than purely stimulus-driven cognitive and behavioral control. The resulting models integrate cognition, decision-making, and behavior in the whole phenotype that may include the genome, physiology, hormonal system, perception, emotions, motivation, and cognition. We conclude that the fundamental state is the global organismic state that includes both physiology and the animal's subjective "mind". The approach provides an avenue for evolutionary understanding of subjective phenomena and self-awareness as evolved mechanisms for adaptive decision-making in natural environments.

show abstract

The principles of goal-directed decision-making: from neural mechanisms to computation and robotics

Abstract: One contribution of 18 to a Theme Issue 'The principles of goal-directed decisionmaking: from neural mechanisms to computation and robotics'.

Cited by 25 publications

References 38 publications

Top-down models in biology: explanation and control of complex living systems above the molecular level

Top-down models in biology: explanation and control of complex living systems above the molecular level

Towards an integration of deep learning and neuroscience

Decision-Making From the Animal Perspective: Bridging Ecology and Subjective Cognition

Contact Info

Product

Resources

About