Dialogue enabling speech-to-text user assistive agent system for hearing-impaired person

Abstract

A novel approach for assisting bidirectional communication between people of normal hearing and hearing-impaired is presented. While the existing hearing-impaired assistive devices such as hearing aids and cochlear implants are vulnerable in extreme noise conditions or post-surgery side effects, the proposed concept is an alternative approach wherein spoken dialogue is achieved by means of employing a robust speech recognition technique which takes into consideration of noisy environmental factors without any attachment into human body. The proposed system is a portable device with an acoustic beamformer for directional noise reduction and capable of performing speech-to-text transcription function, which adopts a keyword spotting method. It is also equipped with an optimized user interface for hearing-impaired people, rendering intuitive and natural device usage with diverse domain contexts. The relevant experimental results confirm that the proposed interface design is feasible for realizing an effective and efficient intelligent agent for hearing-impaired.

Intelligent speech technologies for transcription, disease diagnosis, and medical equipment interactive control in smart hospitals: A review

“…(b) Smart glasses prototype [ 204 ]. (c) A user interfaces for the hearing-impaired person [ 205 ]. (d) Preconditioning trainer [ 207 ].…”

“…23 (c), Lee et al designed an assistive agent system to help the hard of hearing person understand others. When the patient is talking to others, the assist device uses IST to recognize other people's speech as text and utilizes speech synthesis technology to convert the text into speech, helping the patient to communicate normally [ 205 , 218 ].…”

An automatic machine translation system for multi-lingual speech to Indian sign language

“…A novel approach for assisting hearing impaired people effectively has been proposed where a bidirectional dialogue enabled virtual agent system has been developed. The system supports the functionality of both speech-to-text and text-to-speech [60] conversions.…”

Assistive technologies for severe and profound hearing loss: Beyond hearing aids and implants

“…However, it was reported that the processing time took up to 10 seconds which indicates its unsuitability for real time communication. Lee et al (2016) proposed a portable device that performs real time speech-to-text transcription with keyword spotting functionality. The performance of the device seems to be promising as it yielded a low rate of word error (20%).…”