Human Interaction Recognition Based on Whole-Individual Detection

Abstract

Human interaction recognition technology is a hot topic in the field of computer vision, and its application prospects are very extensive. At present, there are many difficulties in human interaction recognition such as the spatial complexity of human interaction, the differences in action characteristics at different time periods, and the complexity of interactive action features. The existence of these problems restricts the improvement of recognition accuracy. To investigate the differences in the action characteristics at different time periods, we propose an improved fusion time-phase feature of the Gaussian model to obtain video keyframes and remove the influence of a large amount of redundant information. Regarding the complexity of interactive action features, we propose a multi-feature fusion network algorithm based on parallel Inception and ResNet. This multi-feature fusion network not only reduces the network parameter quantity, but also improves the network performance; it alleviates the network degradation caused by the increase in network depth and obtains higher classification accuracy. For the spatial complexity of human interaction, we combined the whole video features with the individual video features, making full use of the feature information of the interactive video. A human interaction recognition algorithm based on whole–individual detection is proposed, where the whole video contains the global features of both sides of action, and the individual video contains the individual detail features of a single person. Making full use of the feature information of the whole video and individual videos is the main contribution of this paper to the field of human interaction recognition and the experimental results in the UT dataset (UT–interaction dataset) showed that the accuracy of this method was 91.7%.

Multiperson interaction recognition in images: A body keypoint based feature image analysis

“…However, this survey was more focused on HRI and limited to motion-based individual action analysis. Multifeature fusion network 34 combined the whole video (all humans) features with the individual video (single) features. However, the network relied on video features and limited to two-person scenario.…”

“…However, this method required multicamera setup to acquire 3D information for training. Another major limitation of these methods 1,10,20,31,34,35 is these are trained on video databases and requires temporal information mainly motion. However, for an image this motion cue is absent.…”

“…Furthermore, we convert these feature values to a feature image before being fed to proposed FCNN instead of feeding whole image to ConvNets. 28,29,34…”

“…They obtained an accuracy of 93.3% and 93.6%, respectively, on SBU data set. In order to provide a visual comparison, we have shown methodologies of two state-of-the-art methods, that is, patch based 10 and whole-individual detection 34 in Figure 10. Furthermore, their confusion matrices along with recall, precision and F-measure compared with FCNN are also shown in Tables 5 and 6.…”

“…In order to perform speed and complexity analysis, we reimplemented two state-of-the-art methods, that is, patch based 10 and whole-individual detection. 34 Patch-based model is a low-level interest point based method, whereas the letter is a deep learning based model. Both of these methods are designed to classify videos and required multiple frames for processing.…”