ETRI-Activity3D: A Large-Scale RGB-D Dataset for Robots to Recognize Daily Activities of the Elderly

Jang, Jongsoo; Kim, Dohyung; Park, Cheonshu; Jang, Minsu; Lee, Jaeyeon; Kim, Jae Hong

doi:10.48550/arxiv.2003.01920

Cited by 3 publications

(12 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the early datasets contain only a few thousands of video samples and under 20 class categories, which allowed studies for hand-crafted methods without the support of deep learning. Large-scale activity datasets emerged along with the advance of data-hungry action recognition methods [7], [19]- [24]. The initial version of Kinetics [19], obtained from YouTube, included 400 activity classes having more than 300K in total.…”

Section: Real-world Activity Datasetsmentioning

confidence: 99%

“…The dataset includes eight fixed viewpoints with arbitrary viewpoints sampled from the entire 360°horizontal directions. There also exist other datasets that target action recognition for specific applications, such as eldercare [24], [25]. ETRI-Activity3D [24] captured elders' ADL from several viewpoints considering mobile robots' heights for care robotic services.…”

Section: Real-world Activity Datasetsmentioning

confidence: 99%

“…The need and importance of vision-based human action recognition (HAR) are growing in a wide range of eldercare services [1], including care robots [2], [3], smart surveillance [4], and health monitoring [5], [6]. Recently, the performance of vision-based HAR has been dramatically improved by deep learning methods [7]- [17], which require large-scale training datasets for accurate action recognition [7], [19]- [24] as mentioned in [18]. Accordingly, to train deep learning models to recognize elders' activities of daily living (ADL), we need large-scale datasets which contain activities acquired under various environments and conditions that we encounter in daily life.…”

Section: Introductionmentioning

confidence: 99%

“…The way or speed of actions may also differ from the elders' since they mostly consist of relatively young subjects' actions. These differences can induce inaccurate action recognition results when a model trained on such datasets is tested on data of elders' ADL [24].…”

Section: Introductionmentioning

confidence: 99%

“…Recently, some datasets of elders' ADL have been publicly available [24], [25]. However, due to the limited data acquisition conditions, they often lack diversity in aspects such as background, camera viewpoint, and lighting condition.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications

Hwang

Jang

Park³

et al. 2020

Preprint

View full text Add to dashboard Cite

To train deep learning models for vision-based action recognition of elders' daily activities, we need large-scale activity datasets acquired under various daily living environments and conditions. However, most public datasets used in human action recognition either differ from or have limited coverage of elders' activities in many aspects, making it challenging to recognize elders' daily activities well by only utilizing existing datasets. Recently, such limitations of available datasets have actively been compensated by generating synthetic data from realistic simulation environments and using those data to train deep learning models. In this paper, based on these ideas we develop ElderSim, an action simulation platform that can generate synthetic data on elders' daily activities. For 55 kinds of frequent daily activities of the elders, ElderSim generates realistic motions of synthetic characters with various adjustable data-generating options, and provides different output modalities including RGB videos, two-and three-dimensional skeleton trajectories. We then generate KIST SynADL, a large-scale synthetic dataset of elders' activities of daily living, from ElderSim and use the data in addition to real datasets to train three state-of-the-art human action recognition models. From the experiments following several newly proposed scenarios that assume different real and synthetic dataset configurations for training, we observe a noticeable performance improvement by augmenting our synthetic data. We also offer guidance with insights for the effective utilization of synthetic data to help recognize elders' daily activities.

show abstract

Section: Real-world Activity Datasetsmentioning

confidence: 99%

Section: Real-world Activity Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications

Hwang

Jang

Park³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Motion recognition using deep convolutional neural network for Kinect-based NAO teleoperation

et al. 2022

View full text Add to dashboard Cite

The capabilities of teleoperated robots can be enhanced with the ability to recognise and reproduce human-like behaviour. The proposed framework presents motion recognition for a Kinect-based NAO teleoperation. It allows the NAO robot to recognise the human motions and act as a human motion imitator. A stable whole-body imitation is still a challenging issue because of the difficulty in dynamic balancing of centre of mass (CoM). In this paper, a novel adaptive balancing technique for NAO (ABTN) is proposed to control the whole body in single as well as double supporting phases. It targets dynamic balancing of the humanoid robot by solving forward kinematics and applying a weighted average of mass with the CoMs of individual links with respect to the previous joint frames, which provides us with the dynamic CoM of the whole body. Our novel approach uses this dynamic CoM and calculates joint angles using proposed pitch and roll control algorithm to keep the dynamic CoM inside the stable region. Additionally, the NAO robot is capable of recognising human motions using the proposed 7-layer one-dimensional convolutional neural network (1D-CNN). To solve the problem of variable length of time sequences, Zero padding is introduced with 1D-CNN. It attains a recognition accuracy of 95% as compared to the hidden Markov model and neural network. The experimental results demonstrate that the developed teleoperation framework is robust and serves as potential support for the development and application of teleoperated robots.

show abstract

In-Home Older Adults’ Activity Pattern Monitoring Using Depth Sensors: A Review

Momin

Sufian

Barman

et al. 2022

Sensors

View full text Add to dashboard Cite

The global population is aging due to many factors, including longer life expectancy through better healthcare, changing diet, physical activity, etc. We are also witnessing various frequent epidemics as well as pandemics. The existing healthcare system has failed to deliver the care and support needed to our older adults (seniors) during these frequent outbreaks. Sophisticated sensor-based in-home care systems may offer an effective solution to this global crisis. The monitoring system is the key component of any in-home care system. The evidence indicates that they are more useful when implemented in a non-intrusive manner through different visual and audio sensors. Artificial Intelligence (AI) and Computer Vision (CV) techniques may be ideal for this purpose. Since the RGB imagery-based CV technique may compromise privacy, people often hesitate to utilize in-home care systems which use this technology. Depth, thermal, and audio-based CV techniques could be meaningful substitutes here. Due to the need to monitor larger areas, this review article presents a systematic discussion on the state-of-the-art using depth sensors as primary data-capturing techniques. We mainly focused on fall detection and other health-related physical patterns. As gait parameters may help to detect these activities, we also considered depth sensor-based gait parameters separately. The article provides discussions on the topic in relation to the terminology, reviews, a survey of popular datasets, and future scopes.

show abstract

ETRI-Activity3D: A Large-Scale RGB-D Dataset for Robots to Recognize Daily Activities of the Elderly

Cited by 3 publications

References 25 publications

ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications

ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications

Motion recognition using deep convolutional neural network for Kinect-based NAO teleoperation

In-Home Older Adults’ Activity Pattern Monitoring Using Depth Sensors: A Review

Contact Info

Product

Resources

About