Will You Ever Become Popular? Learning to Predict Virality of Dance Clips

Abstract

Dance challenges are going viral in video communities like TikTok nowadays. Once a challenge becomes popular, thousands of short-form videos will be uploaded within a couple of days. Therefore, virality prediction from dance challenges is of great commercial value and has a wide range of applications, such as smart recommendation and popularity promotion. In this article, a novel multi-modal framework that integrates skeletal, holistic appearance, facial and scenic cues is proposed for comprehensive dance virality prediction. To model body movements, we propose a pyramidal skeleton graph convolutional network (PSGCN) that hierarchically refines spatio-temporal skeleton graphs. Meanwhile, we introduce a relational temporal convolutional network (RTCN) to exploit appearance dynamics with non-local temporal relations. An attentive fusion approach is finally proposed to adaptively aggregate predictions from different modalities. To validate our method, we introduce a large-scale viral dance video (VDV) dataset, which contains over 4,000 dance clips of eight viral dance challenges. Extensive experiments on the VDV dataset well demonstrate the effectiveness of our approach. Furthermore, we show that short video applications such as multi-dimensional recommendation and action feedback can be derived from our model.

Classification of dances using AlexNet, ResNet18 and SqueezeNet1_0

“…Utilizing deep learning models with dance videos has also been performed by several authors. Wang et al [16] proposed the used of a dataset of viral dance videos to predict the virality of a dance video. The authors introduced a relational temporal convolutional network (RTCN) for performing viral predictions of a dance video by incorporating the capture of temporal dynamics from the appearance of the video.…”

“…The authors introduced a relational temporal convolutional network (RTCN) for performing viral predictions of a dance video by incorporating the capture of temporal dynamics from the appearance of the video. Based on their study, factors such as facial, scenic, and holistic appearance of a dance video is considered an important aspect that in virality prediction of a dance video [16]. In our method, we adapt a different approach from the mentioned works by utilizing a dance video dataset with deep learning models to incorporate multiple frames of a dance video to classify the types of dances and for measuring the accuracy of the dance video to how similar the dance video is to others of its type.…”