Human Behavior Recognition Method based on Two-layer LSTM Network with Attention Mechanism

Abstract

Aiming at the problem that the exiting human skeleton-based action recognition methods cannot fully extract the relevant information before and after the action, resulting in low utilization efficiency of skeleton points, we propose a two-layer LSTM (long short term memory) network with attention mechanism. The network has two layers, the first LSTM network is used for skeleton coding and initialization of system storage units and the second LSTM network integrates attention mechanism to further process the data of the first layer network. An algorithm is designed to assign different weights to skeleton points according to the importance of human body, which greatly increases the recognition accuracy. Action classification is accomplished by multiple support vector machines. Through training and testing, the average recognition rate of 98.5% is achieved on KTH dataset. The experimental result shows that the proposed method is effective in human behavior recognition.

Research on micro-expression recognition method based on attention mechanism

“…In this paper, a self-supervised method [4] is used to obtain the forward and reverse optical flow estimates in micro-expression video sequences, where three consecutive frames of images can be extracted to one frame of forward optical flow and one frame of reverse optical flow. Figure 2 shows a 10-frame image sequence and the corresponding extracted 16-frame optical flow sequence with half of the forward optical flow and half of the reverse optical flow [9].…”

Deep Learning Approach for Human Action Recognition Using a Time Saliency Map Based on Motion Features Considering Camera Movement and Shot in Video Image Sequences

“…Shi and Jung [38] used a slow-fast network structure for action detection, which allowed the system to respond differently to slow and fast motions. Gao et al [39] extracted the silhouette region of the image according to the background and then specified the skeleton in different image frames using an LSTM. Wang et al [40] used a dense optical flow field to detect motion in video image sequences and submitted the extracted features to a DNN.…”