Multimodal Information Fusion for Automatic Aesthetics Evaluation of Robotic Dance Poses

Creating a Computable Cognitive Model of Visual Aesthetics for Automatic Aesthetics Evaluation of Robotic Dance Poses

“…Based on different research purposes, aesthetic evaluations are carried out in terms of three aspects: dance pose, dance motion, and dance works [17]. The existing aesthetics evaluation approach to robotic dance poses includes human subjective aesthetics [12][13][14] and machine-learning-based methods [15][16][17].…”

“…In order to train a suitable machine aesthetic model and solve the problem of the automatic aesthetics evaluation of robotic dance poses (mentioned in Section 1), we presented two different approaches in [16,17] based on multi-modal information fusion (visual and non-visual information of robots). Three kinds of features (joint feature, region shape feature, and contour shape feature) of robotic dance poses were extracted and fused to become a mixed feature, and then inputted into several machine-learning methods for training [16]. The best machine aesthetic model obtained an 81.6% correct ratio on AD Tree.…”

“…These mirror images are generally RGB images. This paper uses the method of color thresholding [16] to implement automatic target location. As a prerequisite of the color thresholding method, a dancing robot should have unique color blocks on the important parts of its body, such as its head, shoulder, hand, foot and leg.…”

“…Based on the sub-image of a robotic dance pose (the result of target segmentation), the following two sub-images are further generated: the shape sub-image in Figure 5d, and the spatial distribution sub-image of color blocks in Figure 5e. To acquire the shape sub-image, we use the shape extraction algorithm in [16]: an RGB color image is converted firstly to a black-and-white image, and the eight-connected breed filling algorithm is then used to fill holes and two further morphological operations (corrode and dilate) are conducted; finally, the shape sub-image is extracted. Moreover, the spatial distribution sub-image of color blocks is extracted by the color thresholding method [16] (mentioned in Section 4.1), and the sub-image of a robotic dance pose (the result of target segmentation) is regarded as the input.…”

“…A robot should understand its own dance poses, which appear in a mirror, and make an automatic aesthetics evaluation of them. Some related literatures include human subjective aesthetics [12][13][14] and machine learning based methods [15][16][17]. However, the automatic aesthetic evaluation approach to robotic dance poses, which builds on the aesthetic cognitive neural model of the human brain, has not been investigated to date.…”