Cooking Behavior with Handling General Cooking Tools based on a System Integration for a Life-sized Humanoid Robot

Abstract

This paper describes a system integration for a life-sized robot working at a kitchen. On cooking tasks, there should be various tools and foods, and cooking table may have reflective surface with blots and scratch. Recognition functions should be robust to noises derived from them. As other problems, cooking behaviors impose motion sequences by using whole body of the robot. For instance, while cutting a vegetable, the robot has to hold one hand against the vegetable even if another hand with a knife should be moved for the cutting. This motion requires to consider full articulation of the robot simultaneously. That is, we have difficulties against both recognition and motion generation. In this paper we propose recognition functions that are to detect kitchen tools such as containers and cutting boards. These functions are improved to overcome the influence of reflective surface, and combination shape model with task knowledge is also proposed. On the other hand, we pointed out the importance of the use of torso joints while dual arm manipulation. Our approach enables the robot to keep manipulability of both arms and viewing field of a head. Based on these products, we also introduce an integrated system incorporating recognition modules and motion generation modules. The effectiveness of the system was proven through some cooking applications.

Towards practical robotic chef: Review of relevant work and future challenges

“…The sausage frying robot used a simple color threshold to assess sausage readiness and color segmentation to find the sausage's position on the grill (Mauch et al, 2017). A humanoid cooking robot used the images from a head‐mounted camera to recognize containers, pans, and pots and acted based on this information (Watanabe et al, 2013). The stir‐fry cooking robot used visual feedback of contents deformation to adjust the movements in real‐time to achieve the desired stir‐fry effect (Liu et al, 2022).…”

“…For example, eggplants (Hayashi et al, 2002) and tomato (Feng et al, 2018) were detected using color segmentation. Similar systems made it to robotic chefs too, for example, a humanoid cooking robot used the head‐mounted camera to recognize containers, pans, and pots based on their edges (Watanabe et al, 2013). Object tracking is also useful for robotic chefs and RGBD cameras were used for tracking pizza (Petit et al, 2017) and apples (Nguyen et al, 2014).…”

Robust planning for multi-stage forceful manipulation

“…Fixture planning often relies on a combination of geometric, force, and friction analyses (Hong Lee and Cutkosky, 1991). There are various methods of fixturing including using clamps (Mitsioni et al, 2019), custom jigs (Levi et al, 2022), or using another robot to directly grasp or grasp via tongs (Watanabe et al, 2013; Stuckler et al, 2016; Zhang et al, 2019). Additionally some strategies, such as fixtureless fixturing and shared grasping, rely on friction and environmental contacts to fixture without additional fixtures or tools (Chavan Dafle and Rodriguez, 2018; Hou et al, 2020).…”

Planning for Multi-stage Forceful Manipulation

“…Fixture planning often relies on a combination of geometric, force and friction analyses [16]. There are various methods of fixturing including using clamps [17], using another robot to directly grasp or grasp via tongs [18], [19], [20], or using the environment and relying on friction, such as in fixtureless fixturing or shared grasping [21], [22]. Within this paper we consider fixturing via grasping and environmental contacts.…”