Sofigait—A Wireless Inertial Sensor-Based Gait Sonification System

Abstract

In this study, a prototype of an inertial sensor-based gait sonification system was tested for the purpose of providing real-time gait feedback on the knee angle. The study consisted of two parts: (1) a comparison of the knee angle measurement to a marker-based 3D optical capturing system (Vicon, Oxford, UK) with N = 24 participants and (2) an evaluation four different sonification feedback versions in an accentuation × pitch (2 × 2) design on a sample of N = 28 participants. For the measurement system comparison, the RMSE was 7.6° ± 2.6° for the left and 6.9° ± 3.1° for the right side. Measurement agreement with bias up to −7.5° ± 6.2° (for maximum knee flexion) was indicated by the Bland–Altmann Method. The SPM revealed significant differences between both measurement systems for the area 45–90% (p < 0.001) (left) and the area between 45% and 80% (p = 0.007) (right). For the sonification perception, the variation of pitch had a significant effect on the perception of pleasantness of the sound. No effect was found for the accentuation of the swing or stance phase.

Ultra-responsive, low-dimensional unfamiliar movement sonification guides unconstrained reaches to invisible targets in 3D space

“…Studies sonifying high DoF movements typically sonify and explore the effects on kinematic variables like velocity (Danna et al 2015;Danna and Velay 2017;Liu et al 2022;, or examine the effectiveness of sonification as rehabilitation using coarse-grained spatial sonifications (Nikmaram et al 2019;Raglio et al 2021). Spatial information about body position (e.g., knee angle) has been sonified in high DoF movements like walking (Linnhoff et al 2022) and squatting (Connor et al 2022), however this work reports only the development of the systems, not results on spatial accuracy. Similar sonifications demonstrated to be effective have been limited to low DoF tasks (Ghai et al 2018).…”

“…This preference derives from ecological approaches to auditory perception (Gaver 1993) and in the sonification literature is motivated by the idea that it's easier to interpret feedback that exploits known associations. However, many studies do employ unfamiliar sonifications, in the form of "pure" tones or modulations of white noise (Effenberg et al 2016;Boyer et al 2017;Ghai et al 2018;Connor et al 2022;Linnhoff et al 2022;Matinfar et al 2023).…”

“…Third, movement sonification systems are built with responsiveness well below the temporal processing potential of audition. Typical sonification systems sample sensors and update auditory feedback at 100Hz (Boyer et al 2013(Boyer et al , 2017Effenberg et al 2016;Schaffert et al 2017;Ahmetovic et al 2019;Fehse et al 2020;Linnhoff et al 2022;Liu et al 2022), or 200Hz or 300Hz (Danna and Velay 2017;Dyer et al 2017;), but we are not aware of any sampling faster than that. As for latency, typical systems run either in the 5-29ms range (Boyer et al 2013;Brückner et al 2014;Schaffert et al 2020) or 30-50ms range (Boyer et al 2017;Linnhoff et al 2022), with some running with 100ms or more (Reh et al 2019;Vidal et al 2020).…”

“…Typical sonification systems sample sensors and update auditory feedback at 100Hz (Boyer et al 2013(Boyer et al , 2017Effenberg et al 2016;Schaffert et al 2017;Ahmetovic et al 2019;Fehse et al 2020;Linnhoff et al 2022;Liu et al 2022), or 200Hz or 300Hz (Danna and Velay 2017;Dyer et al 2017;), but we are not aware of any sampling faster than that. As for latency, typical systems run either in the 5-29ms range (Boyer et al 2013;Brückner et al 2014;Schaffert et al 2020) or 30-50ms range (Boyer et al 2017;Linnhoff et al 2022), with some running with 100ms or more (Reh et al 2019;Vidal et al 2020). However, audition has millisecond temporal resolution, with measured gap detection and order sensitivity down to 1-3ms (Green 1971;Fitzgibbons 1983Fitzgibbons , 1984.…”

“…This algorithm converts each sensor's gyroscope readings into quaternions measuring rotation from the position at the start of motion, within the coordinate system defined by that sensor's axes in that initial position. We did not align the readings from the different sensors into a common reference frame, because this is difficult to do accurately (Linnhoff et al 2022). Instead, we wrote a sonification that does not use movement variables (e.g., joint angel or 3D spatial position) requiring frame alignment.…”