A Crowdsourcing-Based Framework for the Development and Validation of Machine Readable Parallel Corpus for Sign Languages

Abstract

Sign languages are used by the deaf and mute community of the world. These are gesture based languages where the subjects use hands and facial expressions to perform different gestures. There are hundreds of different sign languages in the world. Furthermore, like natural languages, there exist different dialects for many sign languages. In order to facilitate the deaf community several different repositories of video gestures are available for many sign languages of the world. These video based repositories do not support the development of an automated language translation systems. This research aims to investigate the idea of engaging the deaf community for the development and validation of a parallel corpus for a sign language and its dialects. As a principal contribution, this research presents a framework for building a parallel corpus for sign languages by harnessing the powers of crowdsourcing with editorial manager, thus it engages a diversified set of stakeholders for building and validating a repository in a quality controlled manner. It further presents processes to develop a word-level parallel corpus for different dialects of a sign language; and a process to develop sentence-level translation corpus comprising of source and translated sentences. The proposed framework has been successfully implemented and involved different stakeholders to build corpus. As a result, a word-level parallel corpus comprising of the gestures of almost 700 words of Pakistan Sign Language (PSL) has been developed. While, a sentencelevel translation corpus comprising of more than 8000 sentences for different tenses has also been developed for PSL. This sentence-level corpus can be used in developing and evaluating machine translation models for natural to sign language translation and vice-versa. While the machine-readable word level parallel corpus will help in generating avatar based videos for the translated sentences in different dialects of a sign language.

Sign language interpreting - relationships between research in different areas - overview

“…In addition, there are many works related to with the analysis of accessibility [19], chaterbots [20], translations of sign languages [21] (e.g. from English (BSL) or American (ALS) to another sign language), or the development of applications related in some way to the implemented project [22]. Such a group in our research was named as other works and requires further analysis.…”

SignUpCrowd: Using Sign-Language as an Input Modality for Microtask Crowdsourcing

“…They focus mainly on how to develop datasets for different sign languages using crowdsourcing techniques. In work by Farooq et al (2021), the authors present a framework for building a parallel corpus for sign languages by exploiting the powers of crowdsourcing. They developed a sentence-level translation corpus comprising more than 8000 sentences for different tenses for Pakistan Sign Language (PSL).…”

“…Existing research at the intersection of crowdsourcing and sign languages focuses on developing ways to build a corpus for various sign languages utilizing crowdsourcing techniques (Riemer Kankkonen et al 2018). Recent work by Farooq et al (2021) investigates the idea of engaging the deaf community for the development and validation of a corpus for a sign language and its dialects. The authors propose a framework for building a corpus for sign languages by leveraging the power of crowdsourcing.…”