Pose Estimation-Assisted Dance Tracking System Based on Convolutional Neural Network

Abstract

In the field of music-driven, computer-assisted dance movement generation, traditional music movement adaptations and statistical mapping models have the following problems: Firstly, the dance sequences generated by the model are not powerful enough to fit the music itself. Secondly, the integrity of the dance movements produced is not sufficient. Thirdly, it is necessary to improve the suppleness and rationality of long-term dance sequences. Fourthly, traditional models cannot produce new dance movements. How to create smooth and complete dance gesture sequences after music is a problem that needs to be investigated in this paper. To address these problems, we design a deep learning dance generation algorithm to extract the association between sound and movement characteristics. During the feature extraction phase, rhythmic features extracted from music and audio beat features are used as musical features, and coordinates of the main points of human bones extracted from dance videos are used for training as movement characteristics. During the model building phase, the model’s generator module is used to achieve a basic mapping of music and dance movements and to generate gentle dance gestures. The identification module is used to achieve consistency between dance and music. The self-encoder module is used to make the audio function more representative. Experimental results on the DeepFashion dataset show that the generated model can synthesize the new view of the target person in any human posture of a given posture, complete the transformation of different postures of the same person, and retain the external features and clothing textures of the target person. Using a whole-to-detail generation strategy can improve the final video composition. For the problem of incoherent character movements in video synthesis, we propose to optimize the character movements by using a generative adversarial network, specifically by inserting generated motion compensation frames into the incoherent movement sequences to improve the smoothness of the synthesized video.

Modified Deep Pattern Classifier on Indonesian Traditional Dance Spatio-Temporal Data

“…The researcher offers valuable insights into the potential of CNN in dance classification, making significant contributions to researchers, choreographers, and dance enthusiasts alike. Furthermore, the findings of this study have substantial implications for automating the analysis and documentation processes associated with cultural dance forms [9]. In other work, CNN structure was optimized for the classification of eight distinct Indian Classical Dances [12].…”

“…Other studies classify videos by selecting the best frames for the training process [21], [22], [23], [24]. In the paper [9], the frame extraction method identifies areas that provide information for each frame and selects frames based on the similarities between those areas. Frames used for subsequent processes are identified using optical flow transfer between frames [21], [22].…”

“…The study utilizes Convolutional Neural Network (CNN) architecture, a renowned approach in the field of image processing and pattern recognition is performed [9]. By employing convolutional layers, CNN can identify essential features such as movement patterns, body shapes, costume textures, and stage layouts that differentiate each type of traditional dance.…”

“…Through an iterative training process, the CNN model continuously improves its identification capabilities by optimizing parameters and minimizing errors [11]. Furthermore, the researchers investigated the application of Deep Convolutional Neural Networks (DCNN) in achieving high accuracy in classifying traditional Indian dance styles [9]. The ResNet50 architecture, in combination with Deep Convolutional Neural Networks (DCNN), was employed.…”