Retrogaming as visual feedback for speech therapy

Abstract

A key problem in speech therapy is the motivation of patients in repetitive vocalization tasks. One important task is the vocalization of vowels. We present a novel solution by incorporating formant speech analysis into retro games to enable intrinsic motivation in performing the vocalization tasks in a fun and accessible manner. The visuals in the retro games also provide a simple and instantaneous feedback mechanism to the patients' vocalization performance. We developed an accurate and efficient formant recognition system to continuously recognize vowel vocalizations in real time. We implemented the system into two games, Speech Invaders and Yak-man, published on the iOS App Store in order to perform an initial public trial. We present the development to inform like-minded researchers who wish to incorporate real-time speech recognition in serious games.

Speech invaders & yak-man

“…paper for the technical details [Tan et al 2014]. By using a simple training step within the game, where the user sounds vowels into the microphone to set particular input controls for the game actions, the engine can accurately detect the specified vowels in English speech.…”

A game based assistive tool for rehabilitation of dysphonic patients

“…However, the people with disabilities may not be able to freely control their hands or feet during the rehabilitation exercises, even not be able to move independently leaving wheelchair [8]. Previous research have proved the usability of the voice interactive serious game in speech therapy [40], for children with autism [13], and parkinsons disease patients [15]. Reducing costs continuously become the pressure of contemporary healthcare systems.…”

Clinical Feedback and Technology Selection of Game Based Dysphonic Rehabilitation Tool

“…Using video game as the vision feedback has been already demonstrated as a useful HCI approach for rehabilitation assistive method [10]. Some previous research have proved the usability of the game in speech therapy [28], for children with autism [5], and parkinsons disease patients [6]. However, none of these researches has considered the success rate of recognition of patients' voice by different pitch estimation algorithms from the view of computer technology.…”