Comparison between Recurrent Networks and Temporal Convolutional Networks Approaches for Skeleton-Based Action Recognition

Abstract

Action recognition plays an important role in various applications such as video monitoring, automatic video indexing, crowd analysis, human-machine interaction, smart homes and personal assistive robotics. In this paper, we propose improvements to some methods for human action recognition from videos that work with data represented in the form of skeleton poses. These methods are based on the most widely used techniques for this problem—Graph Convolutional Networks (GCNs), Temporal Convolutional Networks (TCNs) and Recurrent Neural Networks (RNNs). Initially, the paper explores and compares different ways to extract the most relevant spatial and temporal characteristics for a sequence of frames describing an action. Based on this comparative analysis, we show how a TCN type unit can be extended to work even on the characteristics extracted from the spatial domain. To validate our approach, we test it against a benchmark often used for human action recognition problems and we show that our solution obtains comparable results to the state-of-the-art, but with a significant increase in the inference speed.

An Advanced Noise Reduction and Edge Enhancement Algorithm

“…Noise usually accompanies images during acquisition or transmission, resulting in contrast reduction, color shift, and poor visual quality. The interference of noise not only contaminates the naturalness of an image, but also damages the precision of various computer vision-based applications, such as semantic segmentation [ 1 , 2 ], motion tracking [ 3 , 4 ], action recognition [ 5 , 6 ], and object detection [ 7 , 8 , 9 , 10 , 11 , 12 ], to name a few. Consequently, noise removal for these applications has attracted great interest as a preprocessing task over the last two decades.…”

Skeleton-Based Spatio-Temporal U-Network for 3D Human Pose Estimation in Video

“…A spatial-temporal two-stream transformer network [ 32 ] is proposed to model dependencies between joints using the Transformer self-attention operator. Additionally, some work [ 34 ] has been done to explore and compare different ways of extracting human pose features, and to extend a TCN-like unit to extract the most relevant spatial and temporal characteristics for a sequence of frames.…”

Prognostics for Electromagnetic Relays Using Deep Learning

“…TCN accounts for the caveats of sequence models, compare RNN e.g., LSTM or Gated-Recurrent-Unit Network (GRU) when learning very long sequences [36]. Advantages are the mitigation of the vanishing/exploding gradient problem when back-propagating through time as often encountered with LSTM; reduction of memory usage, training and inference time over traditional RNN architectures [37]; compared to LSTM, TCN also requires less trainable parameters to store intermediate results [35]. To elaborate, 1D-convolution adopted in TCN shares the learned filters across the entire input feature map of length l per input channel c. This can be attributed to the parallelism of the convolution operation.…”

“…TCN, initially presented by [35] addresses above shortcomings. TCN performs dilated, causal convolution -transforming CNN to highly efficient, auto-regressive models as evidenced by [35]- [37]. Unlike e.g., LSTM, TCN is able to be trained on input sequences, irrespective of the length as the number of trainable parameters per layer only depends on the number of input features, filters and the kernel-size.…”