Learning relational affordance models for robots in multi-object manipulation tasks

Moldovan, Bogdan; Moreno, Plínio; Otterlo, Martijn van; Santos-Victor, José; Raedt, Luc De

doi:10.1109/icra.2012.6225042

Cited by 84 publications

(116 citation statements)

References 19 publications

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…It has also been used to learn the dynamics of an object with a single, constant contact (such as pole balancing) (Schaal 1997;Atkeson and Schaal 1997). Finally, there has been work on affordance learning, and work on identifying which variables are relevant to predicting object motion (Montesano et al 2008;Moldovan et al 2012;Hermans et al 2011;Fitzpatrick et al 2003;Ridge et al 2010;Kroemer and Peters 2014). The restriction of these papers is that they make qualitative predictions of object motion, such as a classification of the type of motion outcome.…”

Section: Related Workmentioning

confidence: 99%

“…Most such models are of qualitative effects (Montesano et al 2008;Moldovan et al 2012;Hermans et al 2011;Fitzpatrick et al 2003;Ridge et al 2010;Kroemer and Peters 2014), although metrically precise models have been learned (Meriçli et al 2014;Scholz and Stilman 2010). These learn action-effect correlations.…”

Section: The Importance Of Prediction For Manipulationmentioning

confidence: 99%

See 1 more Smart Citation

Learning modular and transferable forward models of the motions of push manipulated objects

et al. 2016

View full text Add to dashboard Cite

The ability to predict how objects behave during manipulation is an important problem. Models informed by mechanics are powerful, but are hard to tune. An alternative is to learn a model of the object's motion from data, to learn to predict. We study this for push manipulation. The paper starts by formulating a quasi-static prediction problem. We then pose the problem of learning to predict in two different frameworks: (i) regression and (ii) density estimation. Our architecture is modular: many simple, object specific, and context specific predictors are learned. We show empirically that such predictors outperform a rigid body dynamics engine tuned on the same data. We then extend the density estimation approach using a product of experts. This allows transfer of learned motion models to objects of novel shape, and to novel actions. With the right representation and learning method, these transferred models can match the prediction performance of a rigid body dynamics engine for novel objects or actions.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: The Importance Of Prediction For Manipulationmentioning

confidence: 99%

Learning modular and transferable forward models of the motions of push manipulated objects

et al. 2016

View full text Add to dashboard Cite

show abstract

“…[7] [8] [9] [10] [12] Parametric Bias Recurrent Neural Network RNNPB [13] RNNPB [13] RNNPB [13] [14] [15] human-object interaction HOI [16] [17] HOI z C ∼ Mult(θ …”

mentioning

confidence: 99%

Integrated Concept of Human Motions and Objects based on Multi-layered Multimodal LDA

Attamimi¹,

Fadlil²,

Abe³

et al. 2014

JRSJ

View full text Add to dashboard Cite

The human understanding of things is based on prediction which is made through concepts formed by categorization of their experience. To mimic this mechanism in robots, multimodal categorization, which enables the robot to form concepts, has been studied. On the other hand, segmentation and categorization of human motions have also been studied to recognize and predict future motions. This paper addresses the issue of how these concepts are integrated to generate higher level concepts and, more importantly, how the higher level concepts affect each lower level concept formation. To this end, we propose multi-layered multimodal latent Dirichlet allocation (mMLDA) to learn and represent the hierarchical structure of concepts. We also examine a simple integration model and compare with the mMLDA. The experimental results reveal that the mMLDA leads to better inference performance and, indeed, forms higher level concepts integrating motions and objects that are necessary for real-world understanding.

show abstract

“…A ProbLog-based method for robotic action recognition is proposed in Moldovan et al (2012). The method employs a relational extension of the affordance models (Gibson 1979) in order to represent multi-object interactions in a scene.…”

mentioning

confidence: 99%

A probabilistic logic programming event calculus

Skarlatidis

Artikis

Filippou

et al. 2014

Theory and Practice of Logic Programming

View full text Add to dashboard Cite

We present a system for recognising human activity given a symbolic representation of video content. The input of our system is a set of time-stamped short-term activities (STA) detected on video frames. The output is a set of recognised long-term activities (LTA), which are predefined temporal combinations of STA. The constraints on the STA that, if satisfied, lead to the recognition of an LTA, have been expressed using a dialect of the Event Calculus. In order to handle the uncertainty that naturally occurs in human activity recognition, we adapted this dialect to a state-of-the-art probabilistic logic programming framework. We present a detailed evaluation and comparison of the crisp and probabilistic approaches through experimentation on a benchmark dataset of human surveillance videos.

show abstract

Learning relational affordance models for robots in multi-object manipulation tasks

Cited by 84 publications

References 19 publications

Learning modular and transferable forward models of the motions of push manipulated objects

Learning modular and transferable forward models of the motions of push manipulated objects

Integrated Concept of Human Motions and Objects based on Multi-layered Multimodal LDA

A probabilistic logic programming event calculus

Contact Info

Product

Resources

About