Learning Relational Navigation Policies

Die Veröffentlichung steht unter folgender Creative Commons Lizenz: Namensnennung -Keine kommerzielle Nutzung -Keine Bearbeitung 2.0 Deutschland http://creativecommons.org/licenses/by-nc-nd/2.0/de/ Abstract Ever since the word "robot" was introduced to the English language by KarelČapek's play "Rossum's Universal Robots" in 1921, robots have been expected to become part of our daily lives. In recent years, robots such as autonomous vacuum cleaners, lawn mowers, and window cleaners, as well as a huge number of toys have been made commercially available. However, a lot of additional research is required to turn robots into versatile household helpers and companions. One of the many challenges is that robots are still very specialized and cannot easily adapt to changing environments and requirements. Since the 1960s, scientists attempt to provide robots with more autonomy, adaptability, and intelligence. Research in this field is still very active but has shifted focus from reasoning based methods towards statistical machine learning. Both navigation (i.e., moving in unknown or changing environments) and motor control (i.e., coordinating movements to perform skilled actions) are important sub-tasks.In this thesis, we will discuss approaches that allow robots to learn motor skills. We mainly consider tasks that need to take into account the dynamic behavior of the robot and its environment, where a kinematic movement plan is not sufficient. The presented tasks correspond to sports and games but the presented techniques will also be applicable to more mundane household tasks. Motor skills can often be represented by motor primitives. Such motor primitives encode elemental motions which can be generalized, sequenced, and combined to achieve more complex tasks. For example, a forehand and a backhand could be seen as two different motor primitives of playing table tennis. We show how motor primitives can be employed to learn motor skills on three different levels. First, we discuss how a single motor skill, represented by a motor primitive, can be learned using reinforcement learning. Second, we show how such learned motor primitives can be generalized to new situations. Finally, we present first steps towards using motor primitives in a hierarchical setting and how several motor primitives can be combined to achieve more complex tasks.To date, there have been a number of successful applications of learning motor primitives employing imitation learning. However, many interesting motor learning problems are high-dimensional reinforcement learning problems which are often beyond the reach of current reinforcement learning methods. We review research on reinforcement learning applied to robotics and point out key challenges and important strategies to render reinforcement learning tractable. Based on these insights, we introduce novel learning approaches both for single and generalized motor skills.For learning single motor skills, we study parametrized policy search methods and introduce a framework of reward-weighted imi...

show abstract

“…Relational supervised setting [Cocora et al, 2006] and to manipulate articulated objects in simulation [Katz et al, 2008].…”

Section: Smart State-action Discretizationmentioning

confidence: 99%

Learning motor skills: from algorithms to robot experiments

Kober

Peters

2014

It - Information Technology

View full text Add to dashboard Cite

show abstract

“…They defined an ontology about typical domestic environments and then generated plans to find unseen objects or type of rooms, e.g., a bedroom. Cocora et al [1] investigated the problem of how to efficiently find the entrance hall in a hotel. They learned a relational navigation policy that utilizes the information about the type of rooms and corridors that are directly connected to the robot's current location.…”

Section: Related Workmentioning

confidence: 99%

“…In recent years, the problem of inferring and utilizing semantic information in the context of mobile robot navigation has gained substantial interest [1], [2], [3], [4], [5], [6], [7]. This is motivated by the observation that mobile robots can benefit from semantic information in various ways and that it helps to more efficiently carry out their tasks.…”

Section: Introductionmentioning

confidence: 99%

Searching for objects: Combining multiple cues to object locations using a maximum entropy model

Joho

Burgard

2010

2010 IEEE International Conference on Robotics and Automation

Self Cite

View full text Add to dashboard Cite

Abstract-In this paper, we consider the problem of how background knowledge about usual object arrangements can be utilized by a mobile robot to more efficiently find an object in an unknown environment. We decompose the action selection problem during the search into two parts. First, we compute a belief over the location of the object and subsequently use the belief to select the next target location the robot should visit. For the inference part, we utilize a maximum entropy model which models the conditional distribution over possible locations of the target object given the observations made so far. The model is based on co-occurrences of objects and object attributes in different spatial contexts. The parameters are learned by maximizing the data likelihood using gradient ascent. We evaluate our approach by simulated search runs based on data obtained from different real-world environments. Our results show a significant improvement over a standard search technique which does not employ domain-specific background knowledge.

show abstract

“…An abstract state is a conjunction of logical atoms (see also Cocora et al (2006)). Consider for example the abstract state q ≡ far(R, R ′ ), behind(R, R ′ ).…”

Section: Relations Over Distancementioning

confidence: 99%

A Probabilistic Relational Model for Characterizing Situations in Dynamic Multi-Agent Systems

Meyer-Delius

Plagemann

Wichert

et al. 2008

Data Analysis, Machine Learning and Applications

Self Cite

View full text Add to dashboard Cite

Abstract. Artificial systems with a high degree of autonomy require reliable semantic information about the context they operate in. However, state interpretation is a difficult task. Interpretations may depend on a history states and there may be more than one valid interpretation. We propose a model for spatio-temporal situations using hidden Markov models based on relational state descriptions, which are extracted from the estimated state of an underlying dynamic system. Our model covers concurrent situations, scenarios with multiple agents, and situations of varying durations. In this work we apply our model to the concrete task of traffic analysis.

show abstract

Learning Relational Navigation Policies

Cited by 20 publications

References 17 publications

Learning motor skills: from algorithms to robot experiments

Learning motor skills: from algorithms to robot experiments

Searching for objects: Combining multiple cues to object locations using a maximum entropy model

A Probabilistic Relational Model for Characterizing Situations in Dynamic Multi-Agent Systems

Contact Info

Product

Resources

About