Extraction and alignment evaluation of motion beats for street dance

Abstract

The coordination between the dancer's movement and the accompaniment of music is an important element of dance performance. In this paper, we propose a system for coordination evaluation of street dance. Given a dance video clip as input, the system first extracts motion beats from the video and then measures how well the motion beats correlate with the music beats. The motion beats are obtained by analyzing the speed and the change of direction of the dancer's movement. Unlike most previous work, which mainly focuses on 2D motion trajectory analysis, our system provides a more efficient and accurate dance movement analysis by using the 3D joint data of the dancer acquired by Kinect. Another distinction of our system is that, for beat correlation, it considers not only the underlying steady beat of music but also the groove pattern that gives the propulsive rhythmic feel of the music. We believe the differentiation of music beats into these two categories leads to a finer dance coordination evaluation. The test video clips for performance evaluation are generated by professional dancers. The average F -score of our system is about 80%.

Weakly-Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation

“…Motion beat is defined as significant changes in movement at regular intervals. An earlier study [1] revealed that motion beat frames occur when the direction of the movement changes; thus, a motion beat occurs when the speed drops to zero. Furthermore, harmony is a fundamental criterion when dancing to music; hence, the music and motion beat should be synchronized.…”

“…Dancing involves significant changes in motion that occur at regular intervals, i.e., a motion beat (see, e.g., [1], [2], [14]); and when dancing to music, the music and motion beat should be synchronized. In earlier studies, the music beat [8] and other musical features [5] were utilized to improve dance generation.…”

“…Dancing is a performing art and expresses meaning or people's emotion [1]. Dance is composed of sequential rhythmical motion units called basic movements (i.e., basic steps) [2].…”

“…Motivated by the benefits of deep learning, in this study, we explore the application of deep learning models in the generation of basic dance steps. Though deep learning offers the benefits mentioned above, applying it to dance generation can be challenging because of the following three main issues: 1) deep learning models may not generate variable-length non-linear sequences [13] such as dance; 2) the given model may not be able to constrain the motion beat [1], [2], [14] to music beat, and 3) the performance of deep learning models is proportional to the number of training datasets; thus, they require large carefully-labeled datasets for good performance [12].…”

“…Then, the model is trained to maximize the distance when the beat is present, i.e., different; otherwise, it minimizes the distance. To constrain the motion beat to the music beat [1], we assume that a music beat may be present when a motion beat appears. To avoid additional hand-crafted annotations and reduce the amount of training data, the contrastive cost function employs weak labels (see [12], [17], [18]) that are generated by motion direction.…”