Learning Epistemic Actions in Model-Free Memory-Free Reinforcement Learning: Experiments with a Neuro-robotic Model

Ognibene, Dimitri; Volpi, Nicola Catenacci; Pezzulo, Giovanni; Baldassare, Gianluca

doi:10.1007/978-3-642-39802-5_17

Cited by 7 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, the vulnerability we have described can be seen as ideally contiguous with those associated with state identification errors [9,87,88,89,90]. Under conditions of the environment in which information about the states is either incomplete or inaccessible, the resulting interaction between state identification and value estimation can cause the creation of fictitious internal states, where addictive behaviours would always be considered as highly rewarding [9].…”

Section: Discussionmentioning

confidence: 99%

Addiction beyond pharmacological effects: The role of environment complexity and bounded rationality

Ognibene

Fiore

2019

Neural Networks

Self Cite

View full text Add to dashboard Cite

Several decision-making vulnerabilities have been identified as underlying causes for addictive behaviours, or the repeated execution of stereotyped actions despite their adverse consequences. These vulnerabilities are mostly associated with brain alterations caused by the consumption of substances of abuse. However, addiction can also happen in the absence of a pharmacological component, such as seen in pathological gambling and videogaming. We use a new reinforcement learning model to highlight a previously neglected vulnerability that we suggest interacts with those already identified, whilst playing a prominent role in non-pharmacological forms of addiction. Specifically, we show that a duallearning system (i.e. combining model-based and model-free) can be vulnerable to highly rewarding, but suboptimal actions, that are followed by a complex ramification of stochastic adverse effects. This phenomenon is caused by the overload of the capabilities of an agent, as time and cognitive resources required for exploration, deliberation, situation recognition, and habit formation, all increase as a function of the depth and richness of detail of an environment. Furthermore, the cognitive overload can be aggravated due to alterations (e.g.

show abstract

Section: Discussionmentioning

confidence: 99%

Addiction beyond pharmacological effects: The role of environment complexity and bounded rationality

Ognibene

Fiore

2019

Neural Networks

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, the work presented in [80] investigated the role of a bottom-up attention component for the adaptation of a top-down component (see also [68]), the one of [77] focused on the advantages of the actionbased memory component for visual search, the one of [81] investigated how an active-vision system can re-adapt to changing tasks, and the one of [78] focussed on the effects of using manipulation actions to gather information from the world (i.e., as epistemic actions [56]). The novelty of this paper with respect to these works is as follows: (a) the clear identification of the four problems illustrated above, stemming from the active vision solution, and their systematic use to analyse the system behaviour and learning processes; (b) the clear identification of the four architectural principles and the systematic study of how they contribute to solve the four problems; (c) a deeper understanding, supported by new data analyses, of how the interplay of the four principles leads to solve the four problems and also generates some interesting emergent properties of the system (for example the developmental history of the system).…”

Section: ) a Bottom-up Attention Component Guiding Attention Onmentioning

confidence: 99%

Ecological Active Vision: Four Bioinspired Principles to Integrate Bottom–Up and Adaptive Top–Down Attention Tested With a Simple Camera-Arm Robot

Ognibene

Baldassare²

2015

IEEE Trans. Auton. Mental Dev.

Self Cite

View full text Add to dashboard Cite

Vision gives primates a wealth of information useful to manipulate the environment, but at the same time it can easily overwhelm their computational resources. Active vision is a key solution found by nature to solve this problem: a limited fovea actively displaced in space to collect only relevant information.Here we highlight that in ecological conditions this solution encounters four problems: (1) the agent needs to learn where to look based on its goals; (2) manipulation causes learning feedback in areas of space possibly outside the attention focus; (3) good visual actions are needed to guide manipulation actions, but only these can generate learning feedback; (4) a limited fovea causes aliasing problems. We then propose a computational architecture ('BITPIC') to overcome the four problems, integrating four bioinspired key ingredients: (1) reinforcement-learning fovea-based top-down attention; (2) a strong vision-manipulation coupling; (3) bottom-up periphery-based attention; (4) a novel action-oriented memory. The system is tested with a simple simulated cameraarm robot solving a class of search-and-reach tasks involving colour-blob 'objects'. The results show that the architecture solves the problems, and hence the tasks, very efficiently, and highlight how the architecture principles can contribute to a full exploitation of the advantages of active vision in ecological conditions.

show abstract

“…Terekhov et al [35] presents the design and performance of a block-modular neural network architecture allowing parts of the existing network to be re-used to solve novel tasks while retaining performance on the original task. Ognibene et al [36] presents theoretical insights into the unveiling of hidden information through epistemic actions and the experimental benefits of using this actively-gathered information in order to efficiently accomplish a seekand-reach task.…”

Section: Research Backgroundsmentioning

confidence: 99%

Using natural language processing to find research topics in Living Machines conferences and their intersections with Bioinspiration & Biomimetics publications

Carniel¹,

Cazenille²,

Dalle³

et al. 2022

Bioinspir. Biomim.

View full text Add to dashboard Cite

The number of published scientific articles is increasing dramatically and makes it difficult to keep track of research topics. This is particularly difficult interdisciplinary research areas where different communities from different disciplines are working together. It would be useful to develop methods to automate the detection of research topics in a research domain. Here we propose a natural language processing (NLP) based method to automatically detect topics in defined corpora. We start by automatically generating a global state of the art of Living Machines conferences. Our NLP-based method classifies all published papers into different clusters corresponding to the research topic published in these conferences. We perform the same study on all papers published in the journals Bioinspiration & Biomimetics and Soft Robotics. In total this analysis concerns 2099 articles. Next, we analyze the intersection between the research themes published in the conferences and the corpora of these two journals. We also examine the evolution of the number of papers per research theme which determines the research trends. Together, these analyses provide a snapshot of the current state of the field, help to highlight open questions, and provide insights into the future.

show abstract

Learning Epistemic Actions in Model-Free Memory-Free Reinforcement Learning: Experiments with a Neuro-robotic Model

Cited by 7 publications

References 24 publications

Addiction beyond pharmacological effects: The role of environment complexity and bounded rationality

Addiction beyond pharmacological effects: The role of environment complexity and bounded rationality

Ecological Active Vision: Four Bioinspired Principles to Integrate Bottom–Up and Adaptive Top–Down Attention Tested With a Simple Camera-Arm Robot

Using natural language processing to find research topics in Living Machines conferences and their intersections with Bioinspiration & Biomimetics publications

Contact Info

Product

Resources

About