Nrityabodha: Towards understanding Indian classical dance using a deep learning approach

Kishore

Mathematical Problems in Engineering

2017

Extracting and recognizing complex human movements from unconstraint online video sequence is an interesting task. In this paper the complicated problem from the class is approached using unconstraint video sequences belonging to Indian classical dance forms. A new segmentation model is developed using discrete wavelet transform and local binary pattern (LBP) features for segmentation. A 2D point cloud is created from the local human shape changes in subsequent video frames. The classifier is fed with 5 types of features calculated from Zernike moments, Hu moments, shape signature, LBP features, and Haar features. We also explore multiple feature fusion models with early fusion during segmentation stage and late fusion after segmentation for improving the classification process. The extracted features input the Adaboost multiclass classifier with labels from the corresponding song (tala). We test the classifier on online dance videos and on an Indian classical dance dataset prepared in our lab. The algorithms were tested for accuracy and correctness in identifying the dance postures.

Section: Literature Surveymentioning

confidence: 99%

Section: Literature Surveymentioning

confidence: 99%

Indian Classical Dance Classification with Adaboost Multiclass Classifier on Multifeature Fusion

Kishore

Mathematical Problems in Engineering

2017

Communications in Computer and Information Science

“…For posture and sequence recognition, it is critical. Several researchers [7,16,22] worked on posture and gesture recognition without worrying about the segmentation. They assume that the video is already segmented.…”

Section: Key Frame Extractionmentioning

confidence: 99%

Recognition of Adavus in Bharatanatyam Dance

Bhuyan

Das

2021

Digital heritage has been a challenging problem to preserve the cultural heritage like a dance. Digitization of dance requires systematic analyses and understanding the semantics. However, while several work has been reported in non-Indian Classical Dance (ICD); only few has been done in the ICD because of its complexity. Bharatanatyam is an important ICD form, and Adavus constitute the primary collections of postures and movements in Bharatanatyam. There are 15 classes of Adavus having 52 sub-classes between them. Adavus are the critical building blocks for Bharatanatyam and critical for understanding its artefacts and semantics. Hence, this paper attempts to recognize Adavus based on the sequence of Key Postures (KPs) that define and constitute them. Kinect captures dance performances. The videos are then manually segmented into sequences of Key Frames (KFs) corresponding to the expected KPs. For recognizing KP, we follow two approaches; a) skeleton videos and angles of skeleton bones as features, b) The HOG features from the RGB frames. In both approaches, we train SVMs and recognize the KPs using them. The classifier generated by SVM predicts the sequence of KPs involved in a given Adavu. Since KPs are the string-like encoding symbols of an Adavu, we use the predicted sequence and master sequence of KPs in Edit Distance to recognize the matching Adavu. We use a pre-existing annotated data set of Natta (with eight variants) and Mettu (with four variants) Adavus to achieve over 99% accuracy.

“…Apart from these complications, the dancer wears complicated dresses with a nice makeup and at times during performance the backgrounds are changing depending on the story which truly makes this an open-ended problem. Mohanty et al [9] highlights the difficulties in using state-of-the-art pose estimation algorithms such as skeleton estimation [10] and pose estimation [11] failing to track the dancers moves in both offline and online videos. Samanta et al [12] used histogram of oriented optical flow (HOOF) features with sparse representations.…”

Section: Literature Reviewmentioning

confidence: 99%

Indian Classical Dance Action Identification and Classification with Convolutional Neural Networks

Kishore

et al. 2018

Advances in Multimedia

Extracting and recognizing complex human movements from unconstrained online/offline video sequence is a challenging task in computer vision. This paper proposes the classification of Indian classical dance actions using a powerful artificial intelligence tool: convolutional neural networks (CNN). In this work, human action recognition on Indian classical dance videos is performed on recordings from both offline (controlled recording) and online (live performances, YouTube) data. The offline data is created with ten different subjects performing 200 familiar dance mudras/poses from different Indian classical dance forms under various background environments. The online dance data is collected from YouTube for ten different subjects. Each dance pose is occupied for 60 frames or images in a video in both the cases. CNN training is performed with 8 different sample sizes, each consisting of multiple sets of subjects. The remaining 2 samples are used for testing the trained CNN. Different CNN architectures were designed and tested with our data to obtain a better accuracy in recognition. We achieved a 93.33% recognition rate compared to other classifier models reported on the same dataset.