Collecting public RGB-D datasets for human daily activity recognition

Abstract

Human daily activity recognition has been a hot spot in the field of computer vision for many decades. Despite best efforts, activity recognition in naturally uncontrolled settings remains a challenging problem. Recently, by being able to perceive depth and visual cues simultaneously, RGB-D cameras greatly boost the performance of activity recognition. However, due to some practical difficulties, the publicly available RGB-D data sets are not sufficiently large for benchmarking when considering the diversity of their activities, subjects, and background. This severely affects the applicability of complicated learning-based recognition approaches. To address the issue, this article provides a large-scale RGB-D activity data set by merging five public RGB-D data sets that differ from each other on many aspects such as length of actions, nationality of subjects, or camera angles. This data set comprises 4528 samples depicting 7 action categories (up to 46 subcategories) performed by 74 subjects. To verify the challengeness of the data set, three feature representation methods are evaluated, which are depth motion maps, spatiotemporal depth cuboid similarity feature, and curvature space scale. Results show that the merged large-scale data set is more realistic and challenging and therefore more suitable for benchmarking.

The KIMORE Dataset: KInematic Assessment of MOvement and Clinical Scores for Remote Monitoring of Physical REhabilitation

“…Generally, the data have been compared with those collected by other kinds of sensors like the inertial sensor in [45] to test multimodal approaches and develop multimodal datasets [46], [47]. In particular, the diffusion of the Kinect sensor and its accessibility, has allowed the creation of different datasets on the basis of gesture, primitive movements, action and activities as in [48] and [49]. This sensor has also been used as a camera for monitoring emotions related to pain in a rehabilitation context [50].…”

Computer Vision-based Human Activity Recognition for Elderly Home Care

“…They also developed a spatio temporal AND-OR graph for recognizing actions. Hanbo Wu et al [15] combined the above three datasets with MSR Daily Activity 3D and Cornell activity data set (CAD) and created a new dataset with 4528 samples of seven major categories. These actions have 46 subcategories and those had been performed by 74 persons.…”

Multimodal fusion for robotics

“…Part II, which is composed of five papers, is related to heterogonous multimodal methods. The first paper 12 assembles a large-scale RGB-D activity data set by merging five public RGB-D data sets that differ from each other on many aspects such as length of actions, nationality of subjects, or camera angles. This data set provides a new benchmark for research purposes.…”