Live assessment of beat tracking for robot audition

Abstract

Abstract-In this paper we propose the integration of an online audio beat tracking system into the general framework of robot audition, to enable its application in musically-interactive robotic scenarios. To this purpose, we introduced a staterecovery mechanism into our beat tracking algorithm, for handling continuous musical stimuli, and applied different multi-channel preprocessing algorithms (e.g., beamforming, ego noise suppression) to enhance noisy auditory signals lively captured in a real environment. We assessed and compared the robustness of our audio beat tracker through a set of experimental setups, under different live acoustic conditions of incremental complexity. These included the presence of continuous musical stimuli, built of a set of concatenated musical pieces; the presence of noises of different natures (e.g., robot motion, speech); and the simultaneous processing of different audio sources on-the-fly, for music and speech. We successfully tackled all these challenging acoustic conditions and improved the beat tracking accuracy and reaction time to music transitions while simultaneously achieving robust automatic speech recognition.

Making a robot dance to diverse musical genre in noisy environments

“…3, and akin to the experiments run in [4], we created four real-world experimental conditions to assess our musical genre recognition systems in incremental levels of noise complexity:…”

“…However, musical genre recognition has never been attempted in musical robotics under real-world scenarios, whereby experiments were yet much restrict to beat tracking [4] and mood classification [11], [12]. Moreover, when dealing with different noise conditions for robot audition, much of the work has been undertaken on environmental sound source identification [13] and speech recognition [3], [14], [15].…”

“…These include the use of multiple acoustic models, trained under different noise conditions, and the use of Sound Source Separation (SSS) to recognize the speech of three different speakers [14]; the use of compensation and adaption methods to reduce the mismatch between the training and test conditions [15]; and the use of ego noise suppression strategies to tackle the unpredictable diffuse noise generated by the robot motion while performing automatic speech recognition [3] or beat tracking [4]. 14-18, 2014.…”

“…We measured the effect of ego-motion noise in the musical genre recognition accuracy by using the same 3 periodic dance motions used in [4]. Each of them was composed of 2 key-poses interpolated (i.e., transited) during motion generation.…”

“…This is difficult because, although offline genre classification for clean music signals is wellstudied [2], live recognition with noisy auditory input has never been attempted. In addition, the robot's own motor noises (the so-called ego noise [3]) during dancing further degrade the music signal [4].…”