Supporting the creative drive: investigating paralinguistic voice as a mode of interaction for artists with upper limb disabilities

Abstract

For people with upper limb disabilities visual art is an important activity that allows for expression of individuality and independence. They show remarkable endurance, patience and determination to adapt their remaining capabilities to create visual art. There are significant advantages of digital technologies in assisting artists with upper limb disabilities. Paralinguistic voice recognition technologies have proven to be a particularly promising mode of interaction. Despite these benefits, technological support for people with upper limb disabilities to create visual art is scarce. This paper reports on a number of case studies of several artists with upper limb disabilities. These case studies illustrate the struggles they face to be creative and also show the significant advantages of digital technologies in assisting such artists. An investigation into people's ability to use the volume of their voice to control cursor movement to create drawings on the screen is also reported. With motivation, training and practise, use of volume to control drawing tasks shows great promise. It is believed that paralinguistic voice has wider implications beyond assisting artists with upper limb disabilities, such as: an alternative mode of interaction for disabled people to perform tasks other than creating visual art, alternative mode of interaction for hands busy environments and as a voice training system for people with speech impairments.

Revealing lecturer’s paralinguistic attribution: How the visual manner contributes to students’ non-cognitive skills

“…Shortly, accord ing to Johnston (2014), th e paralinguistic attributions or non-verbal signs constituted to the users' physical posture, head movement, eye gaze, facial exp ression, and lips setting. Meanwhile, Perera, Eales, and Blashki (2009) addressed that the paralinguistic attribution qualities importantly allo w the expression and engagement. These prove through the expressive vocalizations, such as laughing and exclaiming, and vocal segregates, such as pitch and volume.…”

sPeAK-MAN

“…While the feedback is different, the underlying approach is still to use the system to accurately evaluate vocal performance and provide unambiguous feedback. Furthermore, there are indications that using these kinds of systems can lead to positive outcomes [1,5,6,7,11]. For example, Perera et al [11] have shown that the use of the voice to create digital art can lead to improved vocal control and [6,7] have provided evidence that abstract audio-visual feedback was effective in encouraging vocalisations of children with Autism Spectrum Disorder (ASD).…”

Retrogaming as visual feedback for speech therapy

“…For example, Perera et al [Perera et al 2008] used screen positions to correspond to vocalizations and showed that using voice to create digital art can lead to improved vocal control. Hailpern et al [Hailpern 2008;Hailpern et al 2009] have provided evidence that abstract audio-visual feedback was effective in encouraging vocalizations of children with Autism Spectrum Disorder (ASD).…”