The MoCA dataset, kinematic and multi-view visual streams of fine-grained cooking actions

Abstract

MoCA is a bi-modal dataset in which we collect Motion Capture data and video sequences acquired from multiple views, including an ego-like viewpoint, of upper body actions in a cooking scenario. It has been collected with the specific purpose of investigating view-invariant action properties in both biological and artificial systems. Besides that, it represents an ideal test bed for research in a number of fields – including cognitive science and artificial vision – and application domains – as motor control and robotics. Compared to other benchmarks available, MoCA provides a unique compromise for research communities leveraging very different approaches to data gathering: from one extreme of action recognition in the wild – the standard practice nowadays in the fields of Computer Vision and Machine Learning – to motion analysis in very controlled scenarios – as for motor control in biomedical applications. In this work we introduce the dataset and its peculiarities, and discuss a baseline analysis as well as examples of applications for which the dataset is well suited.

The VISTA datasets, a combination of inertial sensors and depth cameras data for activity recognition

“…Atomic actions 12 are movements of a person describing a certain motion that can be part of more complex activities, whereas gestures are considered as primitive movements of the body. Activity is hierarchically defined as a sequence of actions 13 . The first part of the experimentation consisted of the training phase, in which each participant had to simulate 10 different actions, sequentially.…”

“…olor video cameras are also often used in human activity recognition to monitor several human people activities. As an example, the Multiview Cooking Actions dataset (MoCA) 5 is a bi-modal dataset in which six VICON infrared cameras are used to collect Motion Capture data and video sequences from multiple views of upper body actions in a cooking scenario. Even if this approach is very interesting, it is based on a very complex and cumbersome system such as the VICON, which cannot be transported and easily installed in a domestic environment.…”

Flipping food during grilling tasks, a dataset of utensils kinematics and dynamics, food pose and subject gaze

“…Indeed, this is a novel dataset: systematically searching Scopus and Web of Science reveals very few datasets including food flipping movements 16 – 18 . They include varied tasks, but each one recorded in a single condition ( e.g ., always with the same food).…”

“…Like ours, their dataset contains position/orientation and force/torque signals. The third dataset 18 includes an action of “turning pancakes” but, once more, without a utensil: throwing the pancake with the pan. The three datasets also differ from ours on the context in which the task was performed.…”

“…Moreover, most published experiments use inorganic food models for the study of cooking tasks, e.g . cardboard/plastic pancakes 18 , 19 , polystyrene cakes or even empty eggshells 16 . However, these are incomplete representations of real cooked food, because they lack the important ability of releasing fluids (water, fat) and hardly reproduce the deformability of food and the stickiness of its compounds ( e.g .…”