Qualitative investigation of the display of speech recognition results for communication with deaf people

Piquard-Kipffer, AgnÃ ̈s; Mella, Odile; Miranda, JÃ©rÃ©my; Jouvet, Denis; Orosanu, Luiza

doi:10.18653/v1/w15-5107

Cited by 14 publications

(4 citation statements)

References 11 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…I think there should be a hierarchy in captions based on the importance to understand the video content. And I should be able to control the level of details in captions... " In terms of caption presentations, our interviewees also complained about existing presentation styles, such as font size [56], font color [60], and underlining [70], which is pre-defined by individual content creators. Five interviewees expressed strong preferences for controlling the changes of caption appearances and the correlation between the changes and the meaning of the changes.…”

Section: Resultsmentioning

confidence: 99%

“…Furthermore, background noise, multi-talker speech, human accent, and disfluent speech may further downgrade the quality of automatic captions [3,18,54]. To make automatic captions work better, prior work explored the approaches such as removing the noise from the environment and changing the appearances of the automatic captions to convey ASR confidence [4] (e.g., alternating the font size [56], font color [60], and underlining [70]). Although these features were studied with DHH participants, they have not been fully integrated into video-sharing platforms, so it is unknown what the platform users will think of them in a practical scenario.…”

Section: Existing Captioning Methodsmentioning

confidence: 99%

See 1 more Smart Citation

An Exploration of Captioning Practices and Challenges of Individual Content Creators on YouTube for People with Hearing Impairments

Li,

Lu,

et al. 2022

Preprint

View full text Add to dashboard Cite

Fig. 1. YouTubers' and Deaf and Hard-of-Hearing audiences' complaints and campaigns on asking for better captions of videos created by individual content creators on video-sharing platforms (e.g., #NoMoreCraptions). Deaf and Hard-of-Hearing (DHH) audiences have long complained about caption qualities for many online videos created by individual content creators on video-sharing platforms (e.g., YouTube). However, there lack explorations of practices, challenges, and perceptions of online video captions from the perspectives of both individual content creators and DHH audiences. In this work, we first explore DHH audiences' feedback on and reactions to YouTube video captions through interviews with 13 DHH individuals, and uncover DHH audiences' experiences, challenges, and perceptions on watching videos created by individual content creators (e.g., manually added caption tags could create additional confidence and trust in caption qualities for DHH audiences). We then discover individual content creators' practices, challenges, and perceptions on captioning their videos (e.g., back-captioning problems) by conducting a YouTube video analysis with 189 captioning-related YouTube videos, followed by a survey with 62 individual content creators. Overall, our findings provide an in-depth understanding of captions generated by individual content creators and bridge the knowledge gap mutually between content creators and DHH audiences on captions. CCS Concepts: • Human-centered computing → Empirical studies in accessibility.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Existing Captioning Methodsmentioning

confidence: 99%

An Exploration of Captioning Practices and Challenges of Individual Content Creators on YouTube for People with Hearing Impairments

Li,

Lu,

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…So far, there have been little research about how to format transcriptions for this particular audience (e.g. highlighting keywords [4], [9]). Furthermore, despite ASR systems good performance, decoding errors are still problematic for captions comprehension [8], especially for students who are deaf who "do not read by 'sounding the words out'".…”

Section: Introductionmentioning

confidence: 99%

Towards Automatic Captioning of University Lectures for French students who are Deaf

Evain

Lecouteux

Portet

et al. 2020

Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility

View full text Add to dashboard Cite

Access to higher education of students who are deaf is below the national average. Recently, there has been a growing number of applications for the automatic transcription of speech, which claim to make everyday speech more accessible to people who are Deaf or Hard-of-Hearing. However, these systems require a good command of the written language, and a significant proportion of the deaf public has low literacy skills. Moreover, we have very little data on how these audiences actually deal with captions. In this paper, we describe the MANES project, whose long-term goal is to assess the usefulness of captioning for the accessibility of lectures by students who are deaf. We present the first technical results of a real-time system to make course captioning suitable for the target audience.CCS Concepts: • Human-centered computing → Accessibility technologies.

show abstract

“…The above-mentioned problems can be solved by involving the assistive technology as it can be used as an interpreter for converting the sign languages into text or speech for better communication between the Deaf community and hearing individuals [12]. Other technologies such as speech technologies can assist in different ways to help people with hearing loss by improving their autonomy [13]. A common example of speech technology is speech recognition, also termed as automatic speech recognition (ASR).…”

Section: Introductionmentioning

confidence: 99%

A Novel Technique for Speech Recognition and Visualization Based Mobile Application to Support Two‐Way Communication between Deaf‐Mute and Normal Peoples

Yousaf

Mehmood

Saba

et al. 2018

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Mobile technology is very fast growing and incredible, yet there are not much technology development and improvement for Deafmute peoples. Existing mobile applications use sign language as the only option for communication with them. Before our article, no such application (app) that uses the disrupted speech of Deaf-mutes for the purpose of social connectivity exists in the mobile market. The proposed application, named as vocalizer to mute (V2M), uses automatic speech recognition (ASR) methodology to recognize the speech of Deaf-mute and convert it into a recognizable form of speech for a normal person. In this work mel frequency cepstral coefficients (MFCC) based features are extracted for each training and testing sample of Deaf-mute speech. The hidden Markov model toolkit (HTK) is used for the process of speech recognition. The application is also integrated with a 3D avatar for providing visualization support. The avatar is responsible for performing the sign language on behalf of a person with no awareness of Deaf-mute culture. The prototype application was piloted in social welfare institute for Deaf-mute children. Participants were 15 children aged between 7 and 13 years. The experimental results show the accuracy of the proposed application as 97.9%. The quantitative and qualitative analysis of results also revealed that face-to-face socialization of Deaf-mute is improved by the intervention of mobile technology. The participants also suggested that the proposed mobile application can act as a voice for them and they can socialize with friends and family by using this app.

show abstract

Qualitative investigation of the display of speech recognition results for communication with deaf people

Cited by 14 publications

References 11 publications

An Exploration of Captioning Practices and Challenges of Individual Content Creators on YouTube for People with Hearing Impairments

An Exploration of Captioning Practices and Challenges of Individual Content Creators on YouTube for People with Hearing Impairments

Towards Automatic Captioning of University Lectures for French students who are Deaf

A Novel Technique for Speech Recognition and Visualization Based Mobile Application to Support Two‐Way Communication between Deaf‐Mute and Normal Peoples

Contact Info

Product

Resources

About