Designing an Interactive Communication Assistance System for Hearing-Impaired College Students Based on Gesture Recognition and Representation

Abstract

Developing a smart classroom can make the modern classroom more efficient and intelligent. Much research has been conducted pertaining to smart classrooms for hearing-impaired college students. However, there have been few significant breakthroughs in mobilizing students’ learning efficiency as measured by information transmission, communication, and interaction in class. This research collects data through nonparticipatory observation and in-depth interviews and analyzes available data on classroom interaction needs of these students. We found that diversified explanations, recordable interactive contents, and interaction between teachers and students could improve the learning effects in the classroom. We also propose a tracking-processing method based on gesture recognition and representation and present a design for a processing system based on AT89C52 microcontroller and Kinect. In this way, sign language can be translated into text and all students can receive the information and participate in the interaction, which greatly improves students’ autonomy and enthusiasm of learning. This design enables deaf students to fully use classroom learning resources, reduces learning time costs, and improves learning efficiency. It can assist teachers in teaching and tutoring students to enhance their experience.

Leveraging Digitalization for Enhanced Publication and Marketing of Sports Services

“…Communication features serve as pivotal conduits for users to access the detailed information they require (Mwombeki, Shidende, & Mselle, 2023;Tang, 2020;Terada et al, 2021;Wei & Han, 2022). Numerous digital platforms offer interactive communication functionalities to facilitate exchanges between service providers and users (Mwombeki, Shidende, & Mselle, 2024;Zhu et al, 2022). The SHC App has been seamlessly integrated with WhatsApp, a widely utilized communication platform in Indonesia.…”

Aquila Based Adaptive Filtering for Hearing Aid with Optimized Performance

“…With the use of the tiny microphone it picks the speech sound and makes it audible and dispatched it to the speaker [5]. The microphone today in the modern world got much smaller and the efficiency is increased [6]. About 10% of individual in the world's population are affected by the hearing loss.…”

A Sign Language Recognition System Applied to Deaf-Mute Medical Consultation

“…The authors of Ref. [ 16 ] proposed a new character input system based on hand tapping gestures that can be used to facilitate human–computer interaction. This system used Kinect.…”