Tac-Trace: A Tangible User Interface-Based Solution for Teaching Shape Concepts to Visually Impaired Children

Abstract

A novel, low-cost, Arabic language tangible user interface (TUI)-based solution for teaching and reinforcing the concepts of planar shapes and their relationships to visually impaired (VI) children is presented. The solution employs a computer vision-based system to track tagged 3D-printed objects which children can manipulate to receive audio instructions and feedback in the context of various activities for learning progressively difficult shape concepts in a systematic manner. A graphical user interface (GUI) is also provided for the teachers to create and manage student accounts, track their students' individual progress and download and print the templates and models of the tangible tokens along with their barcodes. The aim is to provide an engaging, accessible solution that would mitigate the demands on the time, efforts and financial resources of the teachers while allowing children to autonomously learn and review shape concepts at their own pace. By utilizing off-the-shelf components for setting up the TUI and providing templates and CAD models for generating the tangible tokens, we have endeavored to make the system affordable and easily reproducible. The system has been developed in consultation with teachers for VI children to ensure its compliance with current teaching and classroom practices. Initial evaluations of the system with VI children and teachers have validated its potential and yielded valuable design recommendations which would not only inform the design of future iterations of our system but also benefit other researchers seeking to build similar systems.

A Narrative Review of the Sociotechnical Landscape and Potential of Computer-Assisted Dynamic Assessment for Children with Communication Support Needs

“…Jafri et al leveraged computer vision-the AI-powered acquisition, processing, and analysis of digital images-to develop a system with 3D-printed objects which children with sight impairments can use for learning about geometric shapes [114]. Automatic object tracking and recognition via digital cameras has become a significantly more powerful technology in recent years, with competing algorithms from major vendors such as Google, Amazon, and Microsoft.…”

Development of a 3D Printing Strategy to Improve the Level of Abstraction in the Generation of Orthogonal Projections

“…Finally, depending on the chosen settings the element is manufactured by the printer using the fused deposition method (Mihasan, 2021). The student, once with the printed solid, can identify the areas in which he had doubts, in this way the error will be fed back without the need for the teacher to be able to gesticulate or orally explain the fault present in the drawing (Jafri et al, 2022). Thus, one of the greatest benefits of the proposed strategy consists in the direct interaction with the solid, the isometric view is a representation of a geometric solid in a plane considering that an angle of 60 grades that's why is not possible to rotate the graph, in that sense using the 3D printed element, the student with the solid could rotate the element from its three main axes from the top, the left side and the front as seen in Figure 4.A Another form of interaction that turns out to be a significant experience for the student consists of taking measurements of the printed object using a vernier, also gaining an additional skill related to metrology (Renner & Griesbeck, 2020), thus, the student is not only limited to obtaining isometric views but could also obtain information such as length and angles for a real solid, an activity that is highly practiced at an industrial level (Figure 4.B).…”