An Approach to Ballet Dance Training through MS Kinect and Visualization in a CAVE Virtual Reality Environment

Abstract

This article proposes a novel framework for the real-time capture, assessment, and visualization of ballet dance movements as performed by a student in an instructional, virtual reality (VR) setting. The acquisition of human movement data is facilitated by skeletal joint tracking captured using the popular Microsoft (MS) Kinect camera system, while instruction and performance evaluation are provided in the form of 3D visualizations and feedback through a CAVE virtual environment, in which the student is fully immersed. The proposed framework is based on the unsupervised parsing of ballet dance movement into a structured posture space using the spherical self-organizing map (SSOM). A unique feature descriptor is proposed to more appropriately reflect the subtleties of ballet dance movements, which are represented as gesture trajectories through posture space on the SSOM. This recognition subsystem is used to identify the category of movement the student is attempting when prompted (by a virtual instructor) to perform a particular dance sequence. The dance sequence is then segmented and cross-referenced against a library of gestural components performed by the teacher. This facilitates alignment and score-based assessment of individual movements within the context of the dance sequence. An immersive interface enables the student to review his or her performance from a number of vantage points, each providing a unique perspective and spatial context suggestive of how the student might make improvements in training. An evaluation of the recognition and virtual feedback systems is presented.

Multimedia Database Retrieval

“…These methods have some limitations in their capability for temporal information analysis, since they use only the existing of the nodes and the frequency of occurrence of the nodes (key postures) on maps. It is evident that the transitions from posture to posture (or from one form in space to next) preserve more temporal information about the dance sequence than the postures (forms in space) do themselves [340]. In order to perform trajectory analysis on the SSOM, in this Chapter, two methods for transition analysis of the SSOM trajectory are presented.…”

“…11.1 [340]. The Microsoft Kinect system provides 20 3D skeleton points to represent each player (student) in the camera's field of view.…”

“…Thus, the collection of states and their layout can be indicative of some meaning for gesture recognition [339,340]. Recent studies have attempted to analyze the temporal data to see how it maps onto selforganizing maps (SOMs) [338,341,352].…”

Modeling spatial relations of human body parts for indexing and retrieving close character interactions

“…Human motion data have been used in a wide range of VR applications, such as virtual training [Pronost et al 2008;Kyan et al 2015] and virtual rehabilitation [Celiktutan et al 2013], for analyzing the performance of the human subject as well as animating virtual avatars to interact with the subject to enhance the realism of the system. In particular, pre-captured human motions can be used as examples to guide the movement of virtual characters in response to the performance of the human subject [Ho et al 2013;Pronost et al 2008].…”

Estudo de representações multidimensionais para segmentação das fases do gesto

“…Como em Kyan et al (2015) onde a angulação em relação ao troncoé usada na representação do posicionamento de mãos, pulsos, cotovelos, ombros, centro dos ombros, quadris, centro dos quadris, joelhos, pés, tornozelos, espinha e cabeça.…”