Hierarchical reinforcement learning as creative problem solving

Colin, Thomas R.; Belpaeme, Tony; Cangelosi, Angelo; Hemion, Nikolas

doi:10.1016/j.robot.2016.08.021

Cited by 19 publications

(13 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Likewise, agents need to explore the environment to find more optimal solutions. A first approach of combining creativity with reinforcement learning shows that creativity offers the potential to explore promising solution spaces whereas traditional methods fail (Colin et al, 2016). We encourage researchers to build upon recent prosocial behaviour studies to deepen our understanding of efficient coordination.…”

Section: Psychologymentioning

confidence: 99%

Multiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches

Wong¹,

Bäck²,

Kononova³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper surveys the field of multiagent deep reinforcement learning. The combination of deep neural networks with reinforcement learning has gained increased traction in recent years and is slowly shifting the focus from single-agent to multiagent environments. Dealing with multiple agents is inherently more complex as (a) the future rewards depend on multiple players' joint actions and (b) the computational complexity of functions increases. We present the most common multiagent problem representations and their main challenges, and identify five research areas that address one or more of these challenges: centralised training and decentralised execution, opponent modelling, communication, efficient coordination, and reward shaping. We find that many computational studies rely on unrealistic assumptions or are not generalisable to other settings; they struggle to overcome the curse of dimensionality or nonstationarity. Approaches from psychology and sociology capture promising relevant behaviours such as communication and coordination. We suggest that, for multiagent reinforcement learning to be successful, future research addresses these challenges with an interdisciplinary approach to open up new possibilities for more human-oriented solutions in multiagent reinforcement learning.

show abstract

Section: Psychologymentioning

confidence: 99%

Multiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches

Wong¹,

Bäck²,

Kononova³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Several authors suggested that modern reinforcement learning algorithms based on MDPs could allow simulation of the creative process in autonomous agents ( Vigorito and Barto, 2008 ; Schmidhuber, 2010 ; Colin et al, 2016 ). Reinforcement learning (RL) resembles the creative process as both involve interaction between a decision-making agent and its dynamic, uncertain environment, when the agent is searching for a solution to a given problem.…”

Section: Creative Processes In Autonomous Robotsmentioning

confidence: 99%

“…This information is used to deduce an optimal policy ( Kober et al, 2013 ). According to Colin et al (2016) , the agent’s policy changes within hierarchical reinforcement learning algorithms resemble the change in strategies that happens during creative processes.…”

Section: Creative Processes In Autonomous Robotsmentioning

confidence: 99%

Educational Robotics and Robot Creativity: An Interdisciplinary Dialogue

et al. 2021

View full text Add to dashboard Cite

There is a growing literature concerning robotics and creativity. Although some authors claim that robotics in classrooms may be a promising new tool to address the creativity crisis in school, we often face a lack of theoretical development of the concept of creativity and the mechanisms involved. In this article, we will first provide an overview of existing research using educational robotics to foster creativity. We show that in this line of work the exact mechanisms promoted by robotics activities are rarely discussed. We use a confluence model of creativity to account for the positive effect of designing and coding robots on students' creative output. We focus on the cognitive components of the process of constructing and programming robots within the context of existing models of creative cognition. We address as well the question of the role of meta-reasoning and emergent strategies in the creative process. Then, in the second part of the article, we discuss how the notion of creativity applies to robots themselves in terms of the creative processes that can be embodied in these artificial agents. Ultimately, we argue that considering how robots and humans deal with novelty and solve open-ended tasks could help us to understand better some aspects of the essence of creativity.

show abstract

“…Instead of autonomously creating novel options, Konidaris et al ( 2011 ) extended this approach by deriving options from segmenting trajectories trained by demonstration. On a more abstract level, Colin et al ( 2016 ) investigated creativity for problem-solving in artificial agents in the context of hierarchical reinforcement learning by emphasizing parallels to psychology. They argue that hierarchical composition of behaviors allows an agent to handle large search spaces in order to exhibit creative behavior.…”

Section: Related Workmentioning

confidence: 99%

Skill Learning by Autonomous Robotic Playing Using Active Learning and Exploratory Behavior Composition

et al. 2020

View full text Add to dashboard Cite

We treat the problem of autonomous acquisition of manipulation skills where problem-solving strategies are initially available only for a narrow range of situations. We propose to extend the range of solvable situations by autonomous playing with the object. By applying previously-trained skills and behaviours, the robot learns how to prepare situations for which a successful strategy is already known. The information gathered during autonomous play is additionally used to learn an environment model. This model is exploited for active learning and the creative generation of novel preparatory behaviours. We apply our approach on a wide range of different manipulation tasks, e.g. book grasping, grasping of objects of different sizes by selecting different grasping strategies, placement on shelves, and tower disassembly. We show that the creative behaviour generation mechanism enables the robot to solve previously-unsolvable tasks, e.g. tower disassembly. We use success statistics gained during real-world experiments to simulate the convergence behaviour of our system. Experiments show that active improves the learning speed by around 9 percent in the book grasping scenario.

show abstract

Hierarchical reinforcement learning as creative problem solving

Abstract: h i g h l i g h t s • Reinforcement learning's option switches are analogous to psychological insight. • Insight and options reveal comparable capabilities for transformational creativity. • Open problems remain: lifelong learning, switching when exploring, option discovery.

Cited by 19 publications

References 56 publications

Multiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches

Multiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches

Educational Robotics and Robot Creativity: An Interdisciplinary Dialogue

Skill Learning by Autonomous Robotic Playing Using Active Learning and Exploratory Behavior Composition

Contact Info

Product

Resources

About