American Sign Language Video Anonymization to Support Online Participation of Deaf and Hard of Hearing Users

Lee, Sooyeon; Glasser, Abraham; Dingman, Becca; Xia, Zhaoyang; Metaxas, Dimitris N.; Neidle, Carol; Huenerfauth, Matt

doi:10.1145/3441852.3471200

Cited by 14 publications

(9 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The approach combines 3D modeling, ML, and linguistic knowledge derived from annotated video corpora to overcome limitations of prior methods due to aspects like large head rotations, image blurring, and occlusions. Applications could include anonymization of sign language videos while retaining the essential linguistic information conveyed via expressions (Lee et al 2021), tools to help sign language learners produce these grammatically meaningful expressions correctly, and tools for annotation of multimedia language data to advance the understanding of how gestures are used in conjunction with speech (Neidle, Sclaroff, and Athitsos 2001;Neidle 2020).…”

Section: Data and Ai Methods For Modeling Facial Expressions In Langu...mentioning

confidence: 99%

“…Applications could include anonymization of sign language videos while retaining the essential linguistic information conveyed via expressions (Lee et al. 2021), tools to help sign language learners produce these grammatically meaningful expressions correctly, and tools for annotation of multimedia language data to advance the understanding of how gestures are used in conjunction with speech (Neidle, Sclaroff, and Athitsos 2001; Neidle 2020).…”

Section: Healthcare/medicine Neuroscience American Sign Language Meta...mentioning

confidence: 99%

See 1 more Smart Citation

Enabling AI innovation via data and model sharing: An overview of the NSF Convergence Accelerator Track D

et al. 2022

Self Cite

View full text Add to dashboard Cite

This article provides a brief overview of 18 projects funded in Track D—Data and Model Sharing to Enable AI Innovation—of the 2020 Cohort of the National Science Foundation's (NSF) Convergence Accelerator (CA) program. The NSF CA is focused on transitioning research to practice for societal impact. The projects described here were funded for one year in phase I of the program, beginning September 2020. Their focus is on delivering tools, technologies, and techniques to assist in sharing data as well as data‐driven models to enable AI innovation. A broad range of domain areas is covered by the funded efforts, spanning across healthcare and medicine, to climate change and disaster, and civil/built infrastructure. The projects are addressing sharing of open as well as sensitive/private data. In September 2021, six of the eighteen projects described here were selected for phase II of the program, as noted in this article.

show abstract

Section: Data and Ai Methods For Modeling Facial Expressions In Langu...mentioning

confidence: 99%

Section: Healthcare/medicine Neuroscience American Sign Language Meta...mentioning

confidence: 99%

Enabling AI innovation via data and model sharing: An overview of the NSF Convergence Accelerator Track D

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…While achieving anonymous written communication online is relatively uncomplicated for users of written languages, such options have not been readily available to users of sign languages. For video-based communication, individuals who use spoken languages can conceal their faces on online video-sharing platforms however sign-language users cannot due to the crucial linguistic information conveyed through facial expressions [8,15,25,39]. Respondents expressed willingness to view videos created by individuals with their faces disguised as long as there was an ethical purpose [39].…”

Section: Anonymization Of Sign Language Videos For Privacy Concernsmentioning

confidence: 99%

“…For video-based communication, individuals who use spoken languages can conceal their faces on online video-sharing platforms however sign-language users cannot due to the crucial linguistic information conveyed through facial expressions [8,15,25,39]. Respondents expressed willingness to view videos created by individuals with their faces disguised as long as there was an ethical purpose [39]. Sign language is a form of visual communication that employs gestures, facial expressions, and body language.…”

Section: Anonymization Of Sign Language Videos For Privacy Concernsmentioning

confidence: 99%

Towards Inclusive Video Commenting: Introducing Signmaku for the Deaf and Hard-of-Hearing

Chen,

Cheng,

Situ

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Previous research underscored the potential of danmaku-a textbased commenting feature on videos-in engaging hearing audiences. Yet, for many Deaf and hard-of-hearing (DHH) individuals, American Sign Language (ASL) takes precedence over English. To improve inclusivity, we introduce "Signmaku, " a new commenting mechanism that uses ASL, serving as a sign language counterpart to danmaku. Through a need-finding study (N=12) and a withinsubject experiment (N=20), we evaluated three design styles: real human faces, cartoon-like figures, and robotic representations. The results showed that cartoon-like signmaku not only entertained but also encouraged participants to create and share ASL comments, with fewer privacy concerns compared to the other designs. Conversely, the robotic representations faced challenges in accurately depicting hand movements and facial expressions, resulting in higher cognitive demands on users. Signmaku featuring real human faces elicited the lowest cognitive load and was the most comprehensible among all three types. Our findings offered novel design implications for leveraging generative AI to create signmaku comments, enriching co-learning experiences for DHH individuals.

show abstract

“…A large body of work has underscored the importance of providing accessible Information and Communication Technologies (ICTs) to facilitate the adoption and use of ICTs for the DHH population (e.g., [15,29,30,39,48,51]).…”

Section: Accessibility Of Information and Communication Technologies ...mentioning

confidence: 99%

Sparkling Silence: Practices and Challenges of Livestreaming Among Deaf or Hard of Hearing Streamers

Cao,

He,

Zhou

et al. 2023

Preprint

View full text Add to dashboard Cite

Understanding livestream platforms' accessibility challenges for minority groups, such as people with disabilities, is critical to increasing the diversity and inclusion of those platforms. While prior work investigated the experiences of streamers with vision or motor loss, little is known about the experiences of deaf or hard of hearing (DHH) streamers who must work with livestreaming platforms that heavily depend on audio. We conducted semi-structured interviews with DHH streamers to learn why they livestream, how they navigate livestream platforms and related challenges. Our findings revealed their desire to break the stereotypes towards the DHH groups via livestream and the intense interplay between interaction methods, such as sign language, texts, lip language, background music, and viewer characteristics. Major accessibility challenges include the lack of real-time captioning, the small sign language reading window, and misinterpretation of sign language. We present design considerations for improving the accessibility of the livestream platforms.CCS Concepts: • Human-centered computing → Human computer interaction (HCI); Empirical studies in HCI .

show abstract

American Sign Language Video Anonymization to Support Online Participation of Deaf and Hard of Hearing Users

Cited by 14 publications

References 39 publications

Enabling AI innovation via data and model sharing: An overview of the NSF Convergence Accelerator Track D

Enabling AI innovation via data and model sharing: An overview of the NSF Convergence Accelerator Track D

Towards Inclusive Video Commenting: Introducing Signmaku for the Deaf and Hard-of-Hearing

Sparkling Silence: Practices and Challenges of Livestreaming Among Deaf or Hard of Hearing Streamers

Contact Info

Product

Resources

About