Human Action Recognition on Cellphone Using Compositional Bidir-LSTM-CNN Networks

Wang, Jiahao; Long, Qiuling; PiRahc,; Liu, Ke-Xuan; Xie, Yingzi

doi:10.2991/cnci-19.2019.95

Cited by 8 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [14], [322], [323], [324], the tri-axial acceleration data was fed to different CNN architectures for HAR. Wang et al [325] proposed a framework consisting of CNN and Bi-LSTM networks to extract spatial and temporal features from the raw acceleration data. Unlike the above-mentioned works, Lu et al [326] utilized a modified Recurrence Plot (RP) [327] to transform the raw triaxial acceleration data into color images, which were then fed to a ResNet for HAR.…”

Section: Acceleration Modalitymentioning

confidence: 99%

Human Action Recognition From Various Data Modalities: A Review

Sun

Rahmani

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

252

135

View full text Add to dashboard Cite

Human Action Recognition (HAR) aims to understand human behavior and assign a label to each action. It has a wide range of applications, and therefore has been attracting increasing attention in the field of computer vision. Human actions can be represented using various data modalities, such as RGB, skeleton, depth, infrared, point cloud, event stream, audio, acceleration, radar, and WiFi signal, which encode different sources of useful yet distinct information and have various advantages depending on the application scenarios. Consequently, lots of existing works have attempted to investigate different types of approaches for HAR using various modalities. In this paper, we present a comprehensive survey of recent progress in deep learning methods for HAR based on the type of input data modality. Specifically, we review the current mainstream deep learning methods for single data modalities and multiple data modalities, including the fusion-based and the co-learning-based frameworks. We also present comparative results on several benchmark datasets for HAR, together with insightful observations and inspiring future research directions.

show abstract

Section: Acceleration Modalitymentioning

confidence: 99%

Human Action Recognition From Various Data Modalities: A Review

Sun

Rahmani

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

252

135

View full text Add to dashboard Cite

show abstract

“…Moreover, IMU sensor data is a good choice for HAR, due to their robustness against viewpoint, occlusion and background variations. Many works of literature Wang et al (2019a) have proposed wearable sensor-based solutions for HAR. However, the accuracy of IMU sensor-based methods is sensitive to the placement on the human body Mukhopadhyay (2014).…”

Section: Related Workmentioning

confidence: 99%

“…Obviously, there is a huge structure divergence between the imu sensor and vision-sensor data. Since the imu sensor data are one-dimensional time-series signals, most of the previous literature utilize 1D-CNN or LSTM network to extract spatial and temporal features of raw imu sensor data Steven Eyobu & Han (2018); Panwar et al (2017); Wang et al (2019a). The visionsensor activity data, however, usually are images or videos with two or more dimensions, which is suitable for 2D-CNN or 3D-CNN to extract visual features Simonyan & Zisserman (2014); Karpathy et al (2014); Sun et al (2017).…”

Section: Introductionmentioning

confidence: 99%

MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition

Gao¹,

Chen²,

Patel³

et al. 2022

Preprint

View full text Add to dashboard Cite

Multimodal sensors (e.g., visual, non-visual, and wearable) provide complementary information to develop robust perception systems for recognizing activities. However, most existing algorithms use dense sampling and heterogeneous subnetwork to extract unimodal features and fuse them at the end of their framework, which causes data redundancy, lack of complementary multimodal information and high computational cost. In this paper, we propose a new novel multimodal neural architecture based on RGB and IMU wearable sensors (e.g., accelerometer, gyroscope) for human activity recognition called Multimodal Temporal Segment Attention Network (MMTSA). MMTSA first employs a multimodal data isomorphism mechanism based on Gramian Angular Field (GAF) and then applies a novel multimodal sparse sampling method to reduce redundancy. Moreover, we propose an inter-segment attention module in MMTSA to fuse multimodal features effectively and efficiently. We demonstrate the importance of imu data imaging and attention mechanism in human activity recognition by rigorous evaluation on three public datasets, and achieve superior improvements (11.13% on the MMAct dataset) than the previous state-of-the-art methods. The code is available at: https://github.com/THU-CS-PI/MMTSA .

show abstract

Predicting Subscriber Usage: Analyzing Multidimensional Time-Series Using Convolutional Neural Networks

Azaria¹,

Gottlieb

2022

Cyber Security, Cryptology, and Machine Learning

View full text Add to dashboard Cite

Human Action Recognition on Cellphone Using Compositional Bidir-LSTM-CNN Networks

Cited by 8 publications

References 13 publications

Human Action Recognition From Various Data Modalities: A Review

Human Action Recognition From Various Data Modalities: A Review

MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition

Predicting Subscriber Usage: Analyzing Multidimensional Time-Series Using Convolutional Neural Networks

Contact Info

Product

Resources

About