Human Activity Recognition Based on Gramian Angular Field and Deep Convolutional Neural Network

Xu, Hongke; Li, Juan; Yuan, Hui; Liu, Qiang; Fan, Shidi; Li, Tiankuo; Sun, Xiaoxiao

doi:10.1109/access.2020.3032699

Cited by 59 publications

(30 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The data was collected by smartphone, smartwatch and bluetooth devices and experimental results show the reliability of the proposed method. In [25], inertial data is converted into GAF images and then a multisensor data fusion network called Fusion-MdkResNet is used which can process data collected by different sensors and fuse data automatically.…”

Section: Related Workmentioning

confidence: 99%

Inertial Sensor Data to Image Encoding for Human Action Recognition

Ahmad

Khan

2021

IEEE Sensors J.

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) are successful deep learning models in the field of computer vision. To get the maximum advantage of CNN model for Human Action Recognition (HAR) using inertial sensor data, in this paper, we use four types of spatial domain methods for transforming inertial sensor data to activity images, which are then utilized in a novel fusion framework. These four types of activity images are Signal Images (SI), Gramian Angular Field (GAF) Images, Markov Transition Field (MTF) Images and Recurrence Plot (RP) Images. Furthermore, for creating a multimodal fusion framework and to exploit activity images, we made each type of activity images multimodal by convolving with two spatial domain filters : Prewitt filter and High-boost filter. ResNet-18, a CNN model, is used to learn deep features from multi-modalities. Learned features are extracted from the last pooling layer of each ResNet and then fused by canonical correlation based fusion (CCF) for improving the accuracy of human action recognition. These highly informative features are served as input to a multiclass Support Vector Machine (SVM). Experimental results on three publicly available inertial datasets show the superiority of the proposed method over the current state-of-the-art.

show abstract

Section: Related Workmentioning

confidence: 99%

Inertial Sensor Data to Image Encoding for Human Action Recognition

Ahmad

Khan

2021

IEEE Sensors J.

View full text Add to dashboard Cite

show abstract

“…The size of the dance action image is 720 × 480 pixel. We make comparison with other three state-of-the-art action recognition methods including GPRAR (Zhang R. et al, 2018), PGCN-TCA (Xu et al, 2020), MVD (Huynh and Alaghband, 2021). GPRAR, a graph convolutional network based pose reconstruction and action recognition for human trajectory prediction.…”

Section: Experiments and Analysis Experiments Environment And Evaluation Indexmentioning

confidence: 99%

Section: Experiments and Analysismentioning

confidence: 99%

A New Residual Dense Network for Dance Action Recognition From Heterogeneous View Perception

et al. 2021

View full text Add to dashboard Cite

At present, part of people's body is in the state of sub-health, and more people pay attention to physical exercise. Dance is a relatively simple and popular activity, it has been widely concerned. The traditional action recognition method is easily affected by the action speed, illumination, occlusion and complex background, which leads to the poor robustness of the recognition results. In order to solve the above problems, an improved residual dense neural network method is used to study the automatic recognition of dance action images. Firstly, based on the residual model, the features of dance action are extracted by using the convolution layer and pooling layer. Then, the exponential linear element (ELU) activation function, batch normalization (BN) and Dropout technology are used to improve and optimize the model to mitigate the gradient disappearance, prevent over-fitting, accelerate convergence and enhance the model generalization ability. Finally, the dense connection network (DenseNet) is introduced to make the extracted dance action features more rich and effective. Comparison experiments are carried out on two public databases and one self-built database. The results show that the recognition rate of the proposed method on three databases are 99.98, 97.95, and 0.97.96%, respectively. It can be seen that this new method can effectively improve the performance of dance action recognition.

show abstract

“…Angular Field and Deep Convolutional Neural Network (5) Hongji et al proposed a new network, Mdk-ResNet, for large-scale time-series human behavior datasets i ncluding the aforementioned WISDM dataset, (4) which has higher recognition accuracy than the conventional methods. Especially for the aforementioned WISDM dataset, (4) the accuracy is 9.88% higher than that of the MLP-based method by Jennifer R et al…”

Section: Human Activity Recognition Based On Gramianmentioning

confidence: 99%