Pseudo 3D Pose Recognition Network

Abstract

Multi-view human pose recognition has been extensively studied in computer vision due to its significant practical implications. Nonetheless, it remains a challenging task to effectively integrate distinctive view-based features and perform thorough qualitative analysis and quantitative evaluations. In this paper, based on an innovative multi-view fusion module and a novel Mutable Scaling Shortcut Connection, a pseudo 3D pose recognition neural network was meticulously crafted. The proposed network framework comprises four modules: Front Residual Module, 3D Convolution Cross View Fusion Module, Rear Residual Module, and Detection Module. The Front Residual Module serves as the head module with incipient pose heatmaps extraction functionality, taking preprocessed images of various views as separate inputs. The 3D Convolution Cross View Fusion Module performs 3D convolution fusion for the heatmaps output from Front Residual Module of each view, enabling the heatmaps to benefit from each other consequently. The Rear Residual Module extracts deeper-level features, and ultimately the Detection Module performs pose classification and recognition. The proposed network can be trained end-to-end and was evaluated with a Self-Built Multi-View pose recognition dataset. Analytical and evaluation approaches were used to explain the contributory effects of the 3D Convolution Cross View Fusion Module, which significantly improve recognition accuracy from approximately 70% to 91%-94% through Feature Aggregation, Strong Interaction Property among views, Sparsity Reduction, and Increasing Euclidean Distance.