The TUM Kitchen Data Set of everyday manipulation activities for motion tracking and action recognition

Abstract

We introduce the publicly available TUM Kitchen Data Set as a comprehensive collection of activity sequences recorded in a kitchen environment equipped with multiple complementary sensors. The recorded data consists of observations of naturally performed manipulation tasks as encountered in everyday activities of human life. Several instances of a table-setting task were performed by different subjects, involving the manipulation of objects and the environment. We provide the original video sequences, fullbody motion capture data recorded by a markerless motion tracker, RFID tag readings and magnetic sensor readings from objects and the environment, as well as corresponding action labels. In this paper, we both describe how the data was computed, in particular the motion tracker and the labeling, and give examples what it can be used for. We present first results of an automatic method for segmenting the observed motions into semantic classes, and describe how the data can be integrated in a knowledge-based framework for reasoning about the observations.

Collecting complex activity datasets in highly rich networked sensor environments

“…The TUM Kitchen data set was recorded for video-based activity recognition [17]. It also contains RFID and reed switch data, but it does not include on-body sensors.…”

A hierarchical representation for human activity recognition with noisy labels

“…Note that both (15) and (16) can be solved efficiently using a variation of (6). The objective function of (14) is minimized using the Concave Convex Procedure (CCCP) [14].…”

“…The TUM-Kitchen dataset [6] is recorded in a home-care scenario where subjects perform a few daily activities in a kitchen. The kitchen is equipped a set of ambient sensors (i.e., multiple RFID tag reader and magnetic sensors) and four static overhead cameras.…”

“…A second issue concerns the labels of the actions and activities in the dataset. The uncertainty of the labels in the CAD120 (also for TUM-Kitchen [6] and CAD-60 [7]) are unknown. We know that labeling data is a very subjective work and sometimes the annotators may make mistakes which add noise into the labels.…”

Snippet Based Trajectory Statistics Histograms for Assistive Technologies

“…Ziebart et al predicts people's future locations [34] and Kitani et al [12] forecasts human actions by considering the physical environment. Other works involving daily activities include daily action classification or summarization by egocentric videos [7,14,17], fall detection [15], and classification of cooking actions [11,21,23,26].…”