Multimodal Art Pose Recognition and Interaction With Human Intelligence Enhancement

Abstract

This paper provides an in-depth study and analysis of human artistic poses through intelligently enhanced multimodal artistic pose recognition. A complementary network model architecture of multimodal information based on motion energy proposed. The network exploits both the rich information of appearance features provided by RGB data and the depth information provided by depth data as well as the characteristics of robustness to luminance and observation angle. The multimodal fusion is accomplished by the complementary information characteristics of the two modalities. Moreover, to better model the long-range temporal structure while considering action classes with sub-action sharing phenomena, an energy-guided video segmentation method is employed. And in the feature fusion stage, a cross-modal cross-fusion approach is proposed, which enables the convolutional network to share local features of two modalities not only in the shallow layer but also to obtain the fusion of global features in the deep convolutional layer by connecting the feature maps of multiple convolutional layers. Firstly, the Kinect camera is used to acquire the color image data of the human body, the depth image data, and the 3D coordinate data of the skeletal points using the Open pose open-source framework. Then, the action automatically extracted from keyframes based on the distance between the hand and the head, and the relative distance features are extracted from the keyframes to describe the action, the local occupancy pattern features and HSV color space features are extracted to describe the object, and finally, the feature fusion is performed and the complex action recognition task is completed. To solve the consistency problem of virtual-reality fusion, the mapping relationship between hand joint point coordinates and the virtual scene is determined in the augmented reality scene, and the coordinate consistency model of natural hand and virtual model is established; finally, the real-time interaction between hand gesture and virtual model is realized, and the average correct rate of its hand gesture reaches 99.04%, which improves the robustness and real-time interaction of hand gesture recognition.

The Psychology Analysis for Post-production of College Students’ Short Video Communication Education Based on Virtual Image and Internet of Things

“…Although from the digital media technology, the current development in China is not lagging behind compared with that of foreign countries. However, in the expression of artistic language, there is still a big gap between the artistic works (Ma C. M. et al, 2021). The current new media art research in China mainly focuses on the field of art theory.…”

Improved spatial–temporal graph convolutional networks for upper limb rehabilitation assessment based on precise posture measurement

“…Kinect-based joint data contains a variety of information, including spatial information between joint nodes and their adjacent nodes, as well as time-domain information between frames. It has been widely used in motion recognition ( Wang et al, 2020 ), gesture recognition ( Ma et al, 2021 ), somatosensory interaction ( Qiao et al, 2022 ), and also has applications in rehabilitation assessment ( Agami et al, 2022 ) proposed a method for generating accurate skeleton data based on the offline fusion of a Kinect 3D video sensor and an electronic goniometer. This method is difficult to measure the patient’s joint angles with the electronic goniometer ( Lee et al, 2018 ) used Kinect v2 and force sensing resistor sensors based on Fugl-Meyer assessment for evaluating upper extremity motor function ( Bai and Song, 2019 ) conducted a preliminary rehabilitation assessment using the first-generation Kinect to measure the joint data of stroke patients, ignoring the drawbacks of a single camera.…”

Design of Emotion Based Face Recognition System Using Convolutional Neural Networks

“…In this examination, many face expression databases like JAFFE, CK, and others are discussed. The research on classifiers gathered from recent papers provides research fellows with a more thorough and dependable grasp of the specific traits of classifiers [8][9][10][11][12][13][14][15].…”