Movement sonification expectancy model: leveraging musical expectancy theory to create movement-altering sonifications

Abstract

When designing movement sonifications, their effect on people's movement must be considered. Recent work has shown how real-time sonification can be designed to alter the way people move. However, the mechanisms through which these sonifications alter people's expectations of their movement is not well explained. This is especially important when considering musical sonifications, to which people bring their own associations and musical expectation, and which can, in turn, alter their perception of the sonification. This paper presents a Movement Expectation Sonification Model, based on theories of motorfeedback and expectation, to explore how musical sonification can impact the way people perceive their movement. Secondly, we present a study that validates the predictions of this model by exploring how harmonic stability within sonification interacts with contextual cues in the environment to impact movement behaviour and perceptions. We show how musical expectancy can be built to either reward or encourage movement, and how such an effect is mediated through the presence of additional cues. This model offers a way for sonification designers to create movement sonifications that not only inform movement but can be used to encourage progress and reward successes.

SoniBand: Understanding the Effects of Metaphorical Movement Sonifications on Body Perception and Physical Activity

“…This is in-line with new theories of emotional experience, seeing emotions as concepts constructed by multiple brain processes and shaped by factors that characterize people's previous experiences and needs [1]. As such it is important to study not just the common patterns but the variability to better understand the factors that contribute to the effects of sonification that started to emerge in [23,42].…”

Precision strength training: Data-driven artificial intelligence approach to strength and conditioning

“…For example, post-surgery physical rehabilitation patients or beginner yoga practitioners, might be happy with a simple lower your arms'-type of verbal feedback [602,661], while others might nd it strenuous and needlessly ambiguous [282]. Correctly executed exercise could be rewarded with a simpler auditory feedback such as a click [215], or by a movement sonication such as experimented in physical rehabiliation [579,711], or with music feedback that was shown recently to improve deadlift technique [499]. In sEMG studies, it was shown tha tfocused verbal feedback had a positive impact on activating triceps brachii more than without feedback [249,604], highlighting the need of some human oversight and encouragement [565,852].…”

Bodies moving with sound: Effects of pitch and musical sounds on body-representations

“…Effects of harmonic content (stability of musical sounds) on bodily movement: Musically resolved (i.e., ending on a perfect or harmonically stable cadence) vs. unresolved (i.e., ending on an imperfect or harmonically unstable cadence) sonifications accompanying arm raise movements lead people to increase the movement amplitude and stretch for longer, potentially due to musical expectation [35].…”

“…-Effects of harmonic content: Musically resolved sonifications accompanying stretching and squat movements increase feelings of reward and achievement, as well as motivation to continue the movement [35,40,51].…”