Learning to place new objects in a scene

Jiang, Yun; Lim, Marcus; Zheng, Changxi; Saxena, Ashutosh

doi:10.1177/0278364912438781

Cited by 124 publications

(102 citation statements)

References 70 publications

Supporting

Mentioning

101

Contrasting

Order By: Relevance

“…To our best knowledge, there is little work about arranging/placing objects in robotics (e.g., [7,30,11,16,15]), and none of these works consider reasonable arrangements for human usage. In recent work, Jiang et al [14], Jiang and Saxena [13] considered hallucinating humans for object placements and later applied similar idea to the task of scene labeling [17].…”

Section: Related Work: Scene Modelingmentioning

confidence: 99%

“…Dataset. We use the same two datasets as in [14,15]: 1) a synthetic dataset consisting of 20 rooms (living rooms, kitchens and offices) and 47 objects from 19 categories (book, clean tool, laptop, monitor, keyboard, mouse, pen, decoration, dishware, pan, cushion, TV, desk light, floor light, utensil, food, shoe, remote control, and phone); 2) five real offices/apartments each of which is asked to arrange 4, 18, 18, 21 and 18 number of objects. Experimental setup.…”

Section: Related Work: Scene Modelingmentioning

confidence: 99%

See 1 more Smart Citation

Infinite Latent Conditional Random Fields for Modeling Environments through Humans

Jiang

Saxena

2013

Robotics: Science and Systems IX

Self Cite

View full text Add to dashboard Cite

Abstract-Humans cast a substantial influence on their environments by interacting with it. Therefore, even though an environment may physically contain only objects, it cannot be modeled well without considering humans. In this paper, we model environments not only through objects, but also through latent human poses and human-object interactions. However, the number of potential human poses is large and unknown, and the human-object interactions vary not only in type but also in which human pose relates to each object.In order to handle such properties, we present Infinite Latent Conditional Random Fields (ILCRFs) that model a scene as a mixture of CRFs generated from Dirichlet processes. Each CRF represents one possible explanation of the scene. In addition to visible object nodes and edges, it generatively models the distribution of different CRF structures over the latent human nodes and corresponding edges. We apply the model to the challenging application of robotic scene arrangement. In extensive experiments, we show that our model significantly outperforms the state-of-the-art results. We further use our algorithm on a robot for placing objects in a new scene.

show abstract

Section: Related Work: Scene Modelingmentioning

confidence: 99%

Section: Related Work: Scene Modelingmentioning

confidence: 99%

Infinite Latent Conditional Random Fields for Modeling Environments through Humans

Jiang

Saxena

2013

Robotics: Science and Systems IX

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the context of post-grasp manipulation, Jiang et al [11] looked at scenes to determine good locations to place objects. However, they did not study how robust the final process of actually placing an object is, which is the subject of our work.…”

Section: Related Workmentioning

confidence: 99%

A Data-Driven Statistical Framework for Post-Grasp Manipulation

Paolini

Rodríguez

Srinivasa

et al. 2013

Experimental Robotics

View full text Add to dashboard Cite

Grasping an object is usually only an intermediate goal for a robotic manipulator. To finish the task, the robot needs to know where the object is in its hand and what action to execute. This paper presents a general statistical framework to address these problems. Given a novel object, the robot learns a statistical model of grasp state conditioned on sensor values. The robot also builds a statistical model of the requirements of the task in terms of grasp state accuracy. Both of these models are constructed by offline experiments. The online process then grasps objects and chooses actions to maximize likelihood of success. This paper describes the framework in detail, and demonstrates its effectiveness experimentally in placing, dropping, and insertion tasks. To construct statistical models, the robot performed over 8000 grasp trials, and over 1000 trials each of placing, dropping and insertion.

show abstract

“…In our recent work [11,12], we proposed a learning algorithm for placing objects stably and in their preferred orientations in a given placing area (see Fig. 1b).…”

Section: Introductionmentioning

confidence: 99%

Hallucinating Humans for Learning Robotic Placement of Objects

Jiang

Saxena

2013

Springer Tracts in Advanced Robotics

Self Cite

View full text Add to dashboard Cite

Abstract. While a significant body of work has been done on grasping objects, there is little prior work on placing and arranging objects in the environment. In this work, we consider placing multiple objects in complex placing areas, where neither the object nor the placing area may have been seen by the robot before. Specifically, the placements should not only be stable, but should also follow human usage preferences. We present learning and inference algorithms that consider these aspects in placing. In detail, given a set of 3D scenes containing objects, our method, based on Dirichlet process mixture models, samples human poses in each scene and learns how objects relate to those human poses. Then given a new room, our algorithm is able to select meaningful human poses and use them to determine where to place new objects. We evaluate our approach on a variety of scenes in simulation, as well as on robotic experiments.

show abstract

Learning to place new objects in a scene

Cited by 124 publications

References 70 publications

Infinite Latent Conditional Random Fields for Modeling Environments through Humans

Infinite Latent Conditional Random Fields for Modeling Environments through Humans

A Data-Driven Statistical Framework for Post-Grasp Manipulation

Hallucinating Humans for Learning Robotic Placement of Objects

Contact Info

Product

Resources

About