Creating Accessible Interactive Audio-Tactile Drawings using Spatial Augmented Reality

Abstract

Figure 1.A teacher draws interactive zones directly on a tactile map with his finger and records the associated audio-feedback using a microphone. Touching the same zone will later launch the recorded audio-feedback. ABSTRACTInteractive tactile graphics have shown a true potential for people with visual impairments, for instance for acquiring spatial knowledge. Until today, however, they are not well adopted in real-life settings (e.g. special education schools). One obstacle consists in the creation of these media, which requires specific skills, such as the use of vector-graphic software for drawing and inserting interactive zones, which is challenging for stakeholders (social workers, teachers, families of people with visual impairments, etc.). We explored how a Spatial Augmented Reality approach can enhance the creation of interactive tactile graphics by sighted users. We developed the system using a participatory design method. A user study showed that the augmented reality device allowed stakeholders (N=28) to create interactive tactile graphics more efficiently than with a regular vector-drawing software (baseline), independently of their technical background.

Tangible Construction Kit for Blind and Partially Sighted Drawers

“…In contrast, 3D structures were more perceptible [4]. Some studies have explored the use of 3D models with audio annotations [12] and small-scale interactive 3D models [14] as alternatives to visual graphics. Furthering the hypothesis of the effectiveness of using 3D models to improve the creation of mental models in blind and partially sighted individuals, our project aims to introduce the idea of "drawing" from a spatial perspective to consider 3D structures as drawings or representations in addition to 2D raised line drawings.…”

“…Current analog tools such as raised-line drawing kits and swell paper [1], as well as digital devices such as TDraw [9] and grid-based drawing [8], reveal innovation in primarily 2D drawing techniques for blind and partially sighted learners. More recently, tactile shape displays [13] and pin arrays, accessible interactive physical objects through embedded electronics [3] and tracking 3D printed objects through computer vision [7,12] show possibilities for threedimensional (3D) design and creation for BPSI. Other studies have also worked on creating accessible interactive audio-tactile drawings through spatial augmented reality [11].…”

Tactile cartography in the digital age: A review and research agenda

“…However, we are beginning to see initial forays into this area. Indeed, tactile map creation is still a specialized skill, although devices such as the augmented reality drawing tool presented by Thevin et al (2019) show promise for further democratizing the process. 2017) -all demonstrate that the creation of tactile media on the part of people who are B/VI is not only possible but crucial for the evolution of the discipline.…”

Recomendações para o design de imagens audiotáteis impressas em 3D com foco na educação inclusiva de cegos

“…Um dos materiais possíveis de serem utilizados para a educação de cegos é a imagem audiotátil, cujo propósito é ser uma imagem acessível por dois canais sensoriais: o tato e a audição. Ele é um material interativo e áudio responsivo (Holloway, Marriott & Butler, 2018;Kolitsky, 2019;Shi et al, 2019;Thevin et al, 2019), onde a superfície tátil possui pontos associados com explicações e/ou descrições em áudio.…”

“…Como vantagens apontadas por pesquisadores em favor do uso das imagens audiotáteis estão: a diminuição do tamanho da imagem (logo, menos uso de material para fabricação, em comparação com imagens somente táteis), já que não é necessária a impressão em Braille; estudantes que não leem em Braille podem se beneficiar (cegos, videntes e outros estudantes com deficiência); as imagens podem ser reconfiguráveis, trocando-se os áudios associados com a superfície tátil (Holloway, Marriott & Butler, 2018;Thevin et al, 2019).…”