Learning sequential constraints of tasks from user demonstrations

Pardowitz, Michael; Zöllner, R.; Dillmann, Rüdiger

doi:10.1109/ichr.2005.1573604

Cited by 21 publications

(14 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The learning system can state different equally valid hypotheses, ranging from the most restricting precedence graph P D = (N, R D ) that only allows the execution order demonstrated, to the most liberal TPG that possesses no sequential dependencies at all and allows the robot to choose the sequence of operations without any constraints. [15] states that the set of valid hypotheses is a version space, partially ordered by the subset-predicate on the sets of precedence relations R. While the learning system can not decide, which task precedence graph from the set of consistent TPG's fits the task best after seeing only a single example, it seems a viable approach to supply it with more sample demonstrations, applying different task execution orders. In order to learn task knowledge from even a single demonstration sufficient for execution but improving the learned task when more knowledge in form of task demonstrations is available, an incremental approach is chosen.…”

Section: Incremental Learning Of Task Precedence Graphsmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised and Incremental Acquisition of and Reasoning on Holistic Task Knowledge forHousehold Robot Companions

Pardowitz

Zöllner

Dillmann

2006

2006 IEEE/RSJ International Conference on Intelligent Robots and Systems

Self Cite

View full text Add to dashboard Cite

Learning tasks from human demonstration is a core feature for household service robots. To increase the utility of future robot servants, the robot should go beyond simply imitating the user's behavior but try to build flexible, extensible and general task knowledge. This requires higher level reasoning methods that allow the robot to consider its task knowledge in a holistic way. In order to cope with the vast datasets of task knowledge databases, these should be structured in a way that reflects the different classes of tasks as well as the specific characteristics of each task. As the complete set of tasks requested by the user can not be built into the robot, this structure should be learned from the database itself.In this paper, a system to record and interpret manipulation task demonstrations is presented. Unsupervised clustering methods on task knowledge are discussed that at the same time find the task class boundaries and the associated characteristics of each task class. These are used to group task demonstrations in the highly anisotropic feature space and to recognize similar demonstrations belonging to the same class of tasks. In each of the found task classes reasoning methods recover the sequential reordering possibilities, representing them in task precedence graphs. This equips the robot with the ability to update and improve its task knowledge without being put in a dedicated learning mode.

show abstract

Section: Incremental Learning Of Task Precedence Graphsmentioning

confidence: 99%

“…Hypotheses on the sequential task structure can be represented as task precedence graphs [15]. A task precedence graph (TPG) for a task T is a directed graph P = (N, R) with N being the set of subtasks o 1 , o 2 , .…”

Section: Incremental Learning Of Task Precedence Graphsmentioning

confidence: 99%

Unsupervised and Incremental Acquisition of and Reasoning on Holistic Task Knowledge forHousehold Robot Companions

Pardowitz

Zöllner

Dillmann

2006

2006 IEEE/RSJ International Conference on Intelligent Robots and Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [6], [9], task precedence graphs are used to model the flexibility in the sequential relationships of the symbols. This approach considers different permutations of the same symbols and does not consider conditional variability.…”

Section: A Related Workmentioning

confidence: 99%

Generalizing topological task graphs from multiple symbolic demonstrations in programming by demonstration (PbD) processes

Abbas

MacDonald

2011

2011 IEEE International Conference on Robotics and Automation

View full text Add to dashboard Cite

Many programming by demonstration methods encode demonstrations into sequences of predefined symbols and then build a generalized task structure such as a topological graph. The longest common subsequence (LCS) algorithm is one of the potential techniques to help build generalized task structures from multiple sequences. However the LCS problem is NP hard, so a couple of suboptimal LCS approaches have been adopted in the past, involving a pair-wise comparison of sequences or a search for the common symbols within a small window. This paper argues that an LCS of multiple sequences results in a better generalization than pairwise comparison, and in many practical situations it is feasible to find an LCS of multiple sequences. So a novel LCS finding algorithm is presented for applications in the programming by demonstration domain. The algorithm has been extensively tested for sequences of random symbols and its application in a path planning example is presented. I. BACKGROUNDService robots are one of the ultimate goals of robotics. Having successful applications in industries, robotics is more focussed to bring intelligent robotic assistance in our daily life. We all have changing and different activities at changing and different work places, and a robot requires different programs for the different activities and work places. It is infeasible for robot developers to foresee all the possible scenarios and preprogram a robot. End-user robot programming is a potential solution to this robot programming problem [1]- [4] i.e. it would be interesting if a robot could be instructed by its users. However robot users are not expected to have sufficient skills to program the robot. Recently, programming by demonstration (PbD) has demonstrated significant potential and gained considerable recognition amongst end-user robot programming approaches.In a PbD process, a robot is instructed through demonstrations of example tasks. The robot records observations through various sensors while the demonstrator performs the example tasks. The recorded demonstrations are then processed to build a generalized task model. Various factors, such as variations in the workspace and flexibility in the selection of actions, induce variability in different demonstrations of the same task. The task generalization process exploits the variability in the demonstrations for inferencing and hence better learning than pure imitation [2], [4]- [6].For efficient learning of task plans consisting of several moves and interactions, many PbD processes encode the demonstrations into sequences of predefined symbols and then a generalized task structure, such as a hierarchy [6] or a topological graph [5], is built from multiple demonstrations. The symbols represent elementary moves and/or interactions (called primitives). The symbols could have parameters such as goto(x,y) and move-forward(1m) [5], [7], resulting in an infinite set of symbols' alphabet for a PbD system. The process of building a task structure representing all the demonstrated sequences is...

show abstract

“…Just the current and the desired contact state descriptions are known here. This distinguishes our work also from [11]. There, sequential constraints of tasks were learned through a Programming by Demonstration approach.…”

Section: Related Workmentioning

confidence: 99%

Contact state based representation of object relations in the environment for dexterous manipulations

Petsch

Burschka

2013

2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics

View full text Add to dashboard Cite

Abstract-Robots which are supposed to replace a human worker need sophisticated manipulation capabilities. Such capabilities are required for demanding everyday tasks as well as for specialized operations. This includes, e.g., dexterous manipulations in robot-assisted minimally invasive surgery.Sophisticated manipulation tasks require not only appropriate physical capabilities from the hardware, but also advanced knowledge about the dexterous manipulation task itself. Hence, we propose an abstract representation of dexterous manipulation knowledge. It is based on a contact state perspective of the environment. Objects in the environment are described relative to their specific type of contact with the environment. This enables a robot-independent reuse and adaption. The task can even be processed if the environment changes (e.g., due to occurrence of obstacles) of if another robot has to be used.Our experiments illustrate the flexibility which the representation provides. Obstacle avoidance and efficiency of the manipulator's motion are taken into account in the scenarios. I. MOTIVATIONSophisticated manipulation capabilities are an important step towards daily use suitability for robots. This poses not only requirements on the physical capabilities of the involved robotic hand and arm. Advanced knowledge about dexterous manipulation is necessary as well. We focus on the representation of such knowledge in this paper. The representation has to enable a successful reuse of the knowledge even if the environment changes. Hence, it should be possible to build up alternative procedures to achieve the desired goal. Moreover, the knowledge representation should be independent of the robot to allow a knowledge usage on different robotic systems. The representation should be easy to use, since a human should be able to command the robot without programming the exact path. The required knowledge for the representation should be extractable from observation. To conclude, a representation at an abstract level is required rather than, e.g., a record of observations from demonstrations.The aim of a task is usually a desired goal state of objects in the environment. The performance of the task requires, then, a modification of the current state of the environment. In general, a state could be related to a physical state of an object. For example, a cup can be empty or it can be filled with coffee. Hence, its physical state (weight) is different. Moreover, the handling properties of an object can differ figure). Of course, one could use a tool, e.g., a can inbetween to achieve the desired contact state (bottom row). The final choice depends on several factors like efficiency (usage of an additional tool requires time capacities) or further tasks (e.g., refilling of the cup).(e.g., tilting of a cup filled with coffee should be avoided). A representation of such properties has already been presented in [1]. We want to go a step further in this paper. We focus on more dexterous tasks, which require a much more detailed knowledge ...

show abstract

Learning sequential constraints of tasks from user demonstrations

Cited by 21 publications

References 6 publications

Unsupervised and Incremental Acquisition of and Reasoning on Holistic Task Knowledge forHousehold Robot Companions

Unsupervised and Incremental Acquisition of and Reasoning on Holistic Task Knowledge forHousehold Robot Companions

Generalizing topological task graphs from multiple symbolic demonstrations in programming by demonstration (PbD) processes

Contact state based representation of object relations in the environment for dexterous manipulations

Contact Info

Product

Resources

About