Augmentative and Alternative Communication (AAC) aims to complement or replace spoken language to compensate for expression difficulties faced by people with speech impairments. Computing systems have been developed to support AAC; however, partially due to technical problems, poor interface, and limited interaction functions, AAC systems are not widespread, adopted, and used, therefore reaching a limited audience. This article proposes a methodology to support AAC for people with motor impairments, using computer vision and machine learning techniques to allow for personalized gestural interaction. The methodology was applied in a pilot system used by both volunteers without disabilities, and by volunteers with motor and speech impairments, to create datasets with personalized gestures. The created datasets and a public dataset were used to evaluate the technologies employed for gesture recognition, namely the Support Vector Machine (SVM) and Convolutional Neural Network (using Transfer Learning), and for motion representation, namely the conventional Motion History Image and Optical Flow-Motion History Image (OF-MHI). Results obtained from the estimation of prediction error using K-fold cross-validation suggest SVM associated with OF-MHI presents slightly better results for gesture recognition. Results indicate the technical feasibility of the proposed methodology, which uses a low-cost approach, and reveals the challenges and specific needs observed during the experiment with the target audience.
Verbal communication is essential for socialization, meaning construction and knowledge sharing in a society. When verbal communication does not occur naturally because of constraints in people’s and environments capabilities, it is necessary to design alternative means. Augmentative and Alternative Communication (AAC) aims to complement or replace speech to compensate difficulties of verbal expression. AAC systems can provide technological support for people with speech disorders, assisting in the inclusion, learning and sharing of experiences. This paper presents a systematic mapping of the literature to identify research initiatives regarding the use of mobile devices and AAC solutions. The search identified 1366 potentially eligible scientific articles published between 2006 and 2016, indexed by ACM, IEEE, Science Direct, and Springer databases and by the SBC Journal on Interactive Systems. From the retrieved papers, 99 were selected and categorized into themes of research interest: games, autism, usability, assistive technology, AAC, computer interfaces, interaction in mobile devices, education, among others. Most of papers (57 out of 99) presented some form of interaction via mobile devices, and 46 papers were related to assistive technology, from which 14 were related to AAC. The results offer an overview on the applied research on mobile devices for AAC, pointing out to opportunities and challenges in this research domain, with emphasis on the need to promoting the use and effective adoption of assistive technology.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.