Robot, organize my shelves! Tidying up objects by predicting user preferences

Abdo, Nichola; Stachniss, Cyrill; Spinello, Luciano; Burgard, Wolfram

doi:10.1109/icra.2015.7139396

Cited by 53 publications

(35 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Related technologies mainly include scene understanding and segmentation [17], object detection [18] and recognition [19], pose estimation [20], camera calibration [21], action plan [22], motion plan [23] and task plan [24]. Based on this technical roadmap, robots are used successfully for grasping objects [1], tidying up objects [6], cleaning up boxes [25], pouring waters [3], opening doors and drawers [4,26], picking fruits and vegetables [5], cooking [27] and so on. Due to the dynamic changes of the surrounding environment, the diversity of objects and the complexity of manipulation tasks, it is becoming harder and harder and even impossible to pre-determine every step in different robotic manipulation tasks.…”

Section: Related Workmentioning

confidence: 99%

“…Robotic manipulation technology has been widely explored by robotics researchers in past decades. It mainly studies how the robot could move and manipulate objects for various purposes by actions including grasping [1], pushing [2], pouring [3], opening [4], picking [5], placing [6] and so on. Robotic manipulation technology does not focus on the specific robot platforms but rather concentrates on the general related methods of the robot interaction with objects [7].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Composable Instructions and Prospection Guided Visuomotor Control for Robotic Manipulation

Shao¹,

Hu²,

Wang³

et al. 2019

IJCIS

View full text Add to dashboard Cite

Deep neural network-based end-to-end visuomotor control for robotic manipulation is becoming a hot issue of robotics field recently. One-hot vector is often used for multi-task situation in this framework. However, it is inflexible using one-hot vector to describe multiple tasks and transmit intentions of humans. This paper proposes a framework by combining composable instructions with visuomotor control for multi-task problems. The framework mainly consists of two modules: variational autoencoder (VAE) networks and long short-term memory (LSTM) networks. Perception information of the environment is encoded by VAE into a small latent space. The embedded perception information and composable instructions are combined by the LSTM module to guide robotic motion based on different intentions. Prospection is also used to learn the purposes of instructions, which means not only predicting the next action but also predicting a sequence of future actions at the same time. To evaluate this framework, a series of experiments are conducted in pick-and-place application scenarios. For new tasks, the framework could obtain a success rate of 91.2%, which means it has a good generalization ability.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Composable Instructions and Prospection Guided Visuomotor Control for Robotic Manipulation

Shao¹,

Hu²,

Wang³

et al. 2019

IJCIS

View full text Add to dashboard Cite

show abstract

“…Robot shelf stacking was part of the Amazon picking challenge in the last iteration [21]. Teaching robots to tidy up shelves was addressed previously by Abdo et al [22], but their focus is on dividing different products into categories, rather than configuring them on a shelf.…”

Section: Related Workmentioning

confidence: 99%

Simultaneous End-User Programming of Goals and Actions for Robotic Shelf Organization

Liang¹,

Pellier²,

Fiorino³

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Arrangement of items on shelves in stores or warehouses is a tedious, repetitive task that can be feasible for robots to perform. The diversity of products that are available in stores and the different setups and preferences of each store makes pre-programming a robot for this task extremely challenging. Instead, our work argues for enabling end-users to customize the robot to their specific objects and setup at deployment time by programming it themselves. To that end, this paper contributes (i) a task representation for shelf arrangements based on a large dataset of grocery store shelf images, (ii) a method for inferring goal configurations from user inputs including demonstrations and direct parameter specifications, and (iii) a system implementation of the proposed approach that allows simultaneously learning task goals and actions. We evaluate our goal inference approach with ten different teaching strategies that combine alternative user inputs in different ways on the large dataset of grocery configurations, as well as with real human teachers through an online user study (N=32). We evaluate our full system implemented on a Fetch mobile manipulator on eight benchmark tasks that demonstrate endto-end programming and execution of shelf arrangement tasks.

show abstract

“…Personalized collaboration is shown in Fiore et al [8], where an object manipulation task is performed jointly by the robot and the user whose preferences are taken into account. Abdo et al [1] predict user preferences to tidy up objects in containers using collaborative filtering based on crowdsourced data and the observations of current dispositions or by querying the user. Although this strategy seems good for the tidying up task, it would not suit to capture the user preferences in an interaction context such as ours.…”

Section: Related Workmentioning

confidence: 99%

“…For the scope of this paper, we will just focus on the steps involving the user (lines 6, 7 and 8 from Alg. 1), and how they can be personalized to different users 1 .…”

Section: The Robot Feeding Processmentioning

confidence: 99%

Personalization Framework for Adaptive Robotic Feeding Assistance

Canal

Alenyà

Torras

2016

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. The deployment of robots at home must involve robots with pre-defined skills and the capability of personalizing their behavior by non-expert users. A framework to tackle this personalization is presented and applied to an automatic feeding task. The personalization involves the caregiver providing several examples of feeding using Learning-byDemostration, and a ProMP formalism to compute an overall trajectory and the variance along the path. Experiments show the validity of the approach in generating different feeding motions to adapt to user's preferences, automatically extracting the relevant task parameters. The importance of the nature of the demonstrations is also assessed, and two training strategies are compared.

show abstract

Robot, organize my shelves! Tidying up objects by predicting user preferences

Cited by 53 publications

References 21 publications

Composable Instructions and Prospection Guided Visuomotor Control for Robotic Manipulation

Composable Instructions and Prospection Guided Visuomotor Control for Robotic Manipulation

Simultaneous End-User Programming of Goals and Actions for Robotic Shelf Organization

Personalization Framework for Adaptive Robotic Feeding Assistance

Contact Info

Product

Resources

About