Learning Food-arrangement Policies from Raw Images with Generative Adversarial Imitation Learning

Matsuoka, Junki; Tsurumine, Yoshihisa; Kwon, Yuhwan; Matsubara, Takamitsu; Shimmura, Takeshi; Kawamura, Sadao

doi:10.1109/ur49135.2020.9144988

Cited by 11 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Expectation Maximization-based Reinforcement learning (RL) was used by others to learn policies for tossing food and to, more generally, allow a robot to acquire new motor skills from Dynamic Movement Primitives encoding the skill demonstration by a human [209]. Inverse RL (specifically, Generative Adversarial Imitation Learning) was also used to plan the arrangement of food in serving plates like experts do [244]. Recurrent Neural Networks (6 fully connected layers and 2 recurrent layers with 30 units each) were used to learn/model the physical dynamics of cutting food (fruits, vegetables, cake) for a Model Predictive Controller [245], and other studies developed procedures fusing RGB-D image analysis (object segmentation, nearest neighbors correspondences) with physical modelling (finite element method) of non-rigid deformable food, like pizza dough, to track the object while it is being elastically deformed and, this way, support complex food manipulation [246].…”

Section: G Artificial Intelligence For Foodservice Roboticsmentioning

confidence: 99%

Towards Foodservice Robotics: A Taxonomy of Actions of Foodservice Workers and a Critical Review of Supportive Technology

Pereira

Bozzato

Dario

et al. 2022

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

Foodservice workers perform several burdensome, tedious, and unsafe tasks that risk their health and well-being. This could be mitigated or even more avoided by using autonomouslyactuated machines. Therefore, this article aims to build the foundation to support the development of a new field of robotics research dedicated to foodservice and with a human/workercentered framework. As so, we introduce a two-level taxonomy of basic actions that compose the physical tasks of foodservice workers; it can guide future studies to design bio-inspired control models for foodservice robots. Actions are clustered in 16 categories according to their purpose and to the handled food. Furthermore, authors make a critical review of single-action equipment (SAE) and advanced equipment (AE) currently available for foodservice, which allowed us to identify opportunities for research. As a result, authors found some categories of actions rarely automated, aimed at i) separating solid-solid food parts, ii) moving food between workstations or independent appliances in the kitchen, iii) introducing food into another solid food or recipient, and iv) other specific actions, e.g. sewing food. In addition, authors discuss the applicability of collaborative robotics and human-robot collaboration to different contexts in foodservice, and show how artificial intelligence is improving the capabilities of SAE and AE and what else it could improve in this context. Note to practitioners -This paper was motivated by a criticalneed in foodservice: the ability to produce consistent and highquality meals ad-hoc, without overloading the workers or harming their health. Robotic and autonomous systems are promising technologies to solve this. However, there is not a unified framework in robotics research focused on the professional foodservice environment. This paper provides two tools for researchers and engineers in this field: (i) a taxonomy of basic actions that foodservice workers perform during their physical tasks; and (ii) a systematic review of mechatronic systems being developed or already in use in foodservice. The taxonomy can be immediately useful to divide research and development by the classes of actions. In addition, we found specific categories of actions that have been rarely automated so far and need further investigation. The results of our review can be readily applied in industry, too: presently, most equipment is a custom-built machine with limited adaptiveness; when systems include industrial robots, cobots are being preferred; the implementation of collaborative operations between humans and robots is not common yet and its applicability may be suitable only for certain contexts; finally, we identify scientific publications introducing

show abstract

Section: G Artificial Intelligence For Foodservice Roboticsmentioning

confidence: 99%

Towards Foodservice Robotics: A Taxonomy of Actions of Foodservice Workers and a Critical Review of Supportive Technology

Pereira

Bozzato

Dario

et al. 2022

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

show abstract

Section: G Artificial Intelligence For Foodservice Roboticsmentioning

confidence: 99%

Towards Foodservice Robotics: a taxonomy of actions of foodservice workers and a critical review of supportive technology

Pereira¹,

Bozzato²,

Dario³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper was motivated by a critical need in foodservice: the ability to produce consistent and high-quality meals ad-hoc, without overloading the workers or harming their health. Robotic and autonomous systems are promising technologies to solve this. However, there is not a unified framework in robotics research focused on the professional foodservice environment. This paper provides two tools for researchers and engineers in this field: (i) a taxonomy of basic actions that foodservice workers perform during their physical tasks; and (ii) a systematic review of mechatronic systems being developed or already in use in foodservice. The taxonomy can be immediately useful to divide research and development by the classes of actions. In addition, we found specific categories of actions that have been rarely automated so far and need further investigation. The results of our review can be readily applied in industry, too: presently, most equipment is a custom-built machine with limited adaptiveness; when systems include industrial robots, cobots are being preferred; the implementation of collaborative operations between humans and robots is not common yet and its applicability may be suitable only for certain contexts; finally, we identify scientific publications introducing adaptive control strategies and movement policies for some actions that can be implemented today to achieve a more robust actuation.

show abstract

“…shredded vegetables). Prior work in robotics has been successfully demonstrated for the manipulation of large pieces of food in the food industry [1], [2]. However, even though a significant number of small pieces of food are used in the industry, their manipulation is relatively unexplored, and their deformable nature and propensity to get entangled, stick and clump makes them difficult to handle.…”

Section: Introductionmentioning

confidence: 99%

Target-mass Grasping of Entangled Food using Pre-grasping & Post-grasping

Takahashi,

Fukaya,

Ummadisingu

2022

Preprint

View full text Add to dashboard Cite

Food packing industries typically use seasonal ingredients with immense variety that factory workers manually pack. For small pieces of food picked by volume or weight that tend to get entangled, stick or clump together, it is difficult to predict how intertwined they are from a visual examination, making it a challenge to grasp the requisite target mass accurately. Workers rely on a combination of weighing scales and a sequence of complex maneuvers to separate out the food and reach the target mass. This makes automation of the process a non-trivial affair. In this study, we propose methods that combines 1) pre-grasping to reduce the degree of the entanglement, 2) post-grasping to adjust the grasped mass using a novel gripper mechanism to carefully discard excess food when the grasped amount is larger than the target mass, and 3) selecting the grasping point to grasp an amount likely to be reasonably higher than target grasping mass with confidence. We evaluate the methods on a variety of foods that entangle, stick and clump, each of which has a different size, shape, and material properties such as volumetric mass density. We show significant improvement in grasp accuracy of userspecified target masses using our proposed methods.

show abstract

Learning Food-arrangement Policies from Raw Images with Generative Adversarial Imitation Learning

Cited by 11 publications

References 7 publications

Towards Foodservice Robotics: A Taxonomy of Actions of Foodservice Workers and a Critical Review of Supportive Technology

Towards Foodservice Robotics: A Taxonomy of Actions of Foodservice Workers and a Critical Review of Supportive Technology

Towards Foodservice Robotics: a taxonomy of actions of foodservice workers and a critical review of supportive technology

Target-mass Grasping of Entangled Food using Pre-grasping & Post-grasping

Contact Info

Product

Resources

About