Learning to place new objects

Jiang, Yun; Zheng, Changxi; Lim, Marcus; Saxena, Ashutosh

doi:10.1109/icra.2012.6224581

Cited by 35 publications

(27 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our best knowledge, there is little work about arranging/placing objects in robotics (e.g., [7,30,11,16,15]), and none of these works consider reasonable arrangements for human usage. In recent work, Jiang et al [14], Jiang and Saxena [13] considered hallucinating humans for object placements and later applied similar idea to the task of scene labeling [17].…”

Section: Related Work: Scene Modelingmentioning

confidence: 99%

Infinite Latent Conditional Random Fields for Modeling Environments through Humans

Jiang

Saxena

2013

Robotics: Science and Systems IX

Self Cite

View full text Add to dashboard Cite

Abstract-Humans cast a substantial influence on their environments by interacting with it. Therefore, even though an environment may physically contain only objects, it cannot be modeled well without considering humans. In this paper, we model environments not only through objects, but also through latent human poses and human-object interactions. However, the number of potential human poses is large and unknown, and the human-object interactions vary not only in type but also in which human pose relates to each object.In order to handle such properties, we present Infinite Latent Conditional Random Fields (ILCRFs) that model a scene as a mixture of CRFs generated from Dirichlet processes. Each CRF represents one possible explanation of the scene. In addition to visible object nodes and edges, it generatively models the distribution of different CRF structures over the latent human nodes and corresponding edges. We apply the model to the challenging application of robotic scene arrangement. In extensive experiments, we show that our model significantly outperforms the state-of-the-art results. We further use our algorithm on a robot for placing objects in a new scene.

show abstract

Section: Related Work: Scene Modelingmentioning

confidence: 99%

Infinite Latent Conditional Random Fields for Modeling Environments through Humans

Jiang

Saxena

2013

Robotics: Science and Systems IX

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the following sections, we first briefly review our potential function that captures the object context-relationship between the object and placing areas [11,12]. Then we discuss how to encode human context (such as human usage preferences and access effort) in our algorithm.…”

Section: Object and Human Contextmentioning

confidence: 99%

“…In our previous work [11,12], we considered only the object context Ψ object (·). In this current work, we primarily use the human context Ψ human (·), and only combine it with some of the object context defined heuristically (see [13]).…”

Section: Human Contextmentioning

confidence: 99%

“…In our own recent work [11,12], we employed 3D stability and geometric features to find stable and preferred placements. However, without taking human context into consideration, the generated strategy was often not good enough.…”

Section: Related Workmentioning

confidence: 99%

“…In our recent work [11,12], we proposed a learning algorithm for placing objects stably and in their preferred orientations in a given placing area (see Fig. 1b).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Hallucinating Humans for Learning Robotic Placement of Objects

Jiang

Saxena

2013

Springer Tracts in Advanced Robotics

Self Cite

View full text Add to dashboard Cite

Abstract. While a significant body of work has been done on grasping objects, there is little prior work on placing and arranging objects in the environment. In this work, we consider placing multiple objects in complex placing areas, where neither the object nor the placing area may have been seen by the robot before. Specifically, the placements should not only be stable, but should also follow human usage preferences. We present learning and inference algorithms that consider these aspects in placing. In detail, given a set of 3D scenes containing objects, our method, based on Dirichlet process mixture models, samples human poses in each scene and learns how objects relate to those human poses. Then given a new room, our algorithm is able to select meaningful human poses and use them to determine where to place new objects. We evaluate our approach on a variety of scenes in simulation, as well as on robotic experiments.

show abstract

Autonomous dishwasher loading from cluttered trays using pre‐trained deep neural networks

Voysey

Thuruthel

Iida

2020

Engineering Reports

View full text Add to dashboard Cite

Autonomous dishwasher loading is a benchmark problem in robotics that highlights the challenges of robotic perception, planning, and manipulation in an unstructured environment. Current approaches resort to a specialized solution, however, these technologies are not viable in a domestic setting. Learning-based solutions seem promising for a general purpose solutions; however, they require large amounts of catered data to be applied in real-world scenarios. This article presents a novel learning-based solution without a training phase using pre-trained object detection networks. By developing a perception, planning, and manipulation framework around an off-the-shelf object detection network, we are able to develop robust pick-and-place solutions that are easy to develop and general purpose requiring only a RGB feedback and a pinch gripper. Analysis of a real-world canteen tray data is first performed and used for developing our in-lab experimental setup. Our results obtained from real-world scenarios indicate that such approaches are highly desirable for plug-and-play domestic applications with limited calibration. All the associated data and code of this work are shared in a public repository.

show abstract

Learning to place new objects

Cited by 35 publications

References 22 publications

Infinite Latent Conditional Random Fields for Modeling Environments through Humans

Infinite Latent Conditional Random Fields for Modeling Environments through Humans

Hallucinating Humans for Learning Robotic Placement of Objects

Autonomous dishwasher loading from cluttered trays using pre‐trained deep neural networks

Contact Info

Product

Resources

About