Real‐time action feature extraction via fast PCA‐Flow

Abstract

Summary Action recognition is a research hotspot in the field of Internet of Things (IoT). Currently, local pixel‐domain spatiotemporal feature extraction methods have reached the state‐of‐the‐art action recognition performance on many challenging datasets. However, the poor computational complexity of these approaches prevents them from scaling up to real‐time applications. For solving this problem, we present a novel real‐time video feature extraction technique by exploiting the fast PCA‐Flow algorithm. Firstly, we down‐sample video images in form of grid. Based on the down‐sampling images, PCA‐Flow algorithm is used to calculate optical flow among adjacent images. The PCA‐Flow matrices are then expanded to the original video image size by using efficient gCLSR super‐resolution method to keep the inherent geometric structure of the optical flow. Finally, we compute action descriptors based on original pixel frames and the enlarged PCA‐Flow images. The proposed approach is validated on three challenging datasets: UCF50, Hollywood2, and HMDB51. Experimental results indicate that the proposed method is more efficient in computation and can achieve competitive quality than the state‐of‐the‐art methods.

Foreword to the special issue of the intelligent systems for the Internet of Things (ISIT2018)

“…multiresolution amplitude feature. This article proposed a new ideal for emotion feature extracting and broaden solution of speech emotion recognition.In the article Real-time action feature extraction via fast PCA-Flow,3 a novel real-time action feature extraction method for human action recognition is provided. Video-based human action recognition can be widely used in network video retrieval, video surveillance analysis, medical video monitoring, and so forth.…”