Quantitative Survey of the State of the Art in Sign Language Recognition

Abstract

This work presents a meta study covering around 300 published sign language recognition papers with over 400 experimental results. It includes most papers between the start of the field in 1983 and 2020. Additionally, it covers a fine-grained analysis on over 25 studies that have compared their recognition approaches on RWTH-PHOENIX-Weather 2014, the standard benchmark task of the field. Research in the domain of sign language recognition has progressed significantly in the last decade, reaching a point where the task attracts much more attention than ever before. This study compiles the state of the art in a concise way to help advance the field and reveal open questions. Moreover, all of this meta study's source data is made public, easing future work with it and further expansion. The analyzed papers have been manually labeled with a set of categories. The data reveals many insights, such as, among others, shifts in the field from intrusive to non-intrusive capturing, from local to global features and the lack of non-manual parameters included in medium and larger vocabulary recognition systems. Surprisingly, RWTH-PHOENIX-Weather with a vocabulary of 1080 signs represents the only resource for large vocabulary continuous sign language recognition benchmarking world wide.

Including Signed Languages in Natural Language Processing

“…The introduction of a continuous signed language benchmark dataset (Forster et al, 2014;, coupled with the advent of deep learning for visual processing, lead to increased efforts to recognize signed expressions from videos. Recent surveys on SLP mostly review these different approaches for sign language recognition developed by the CV community (Koller, 2020;Rastgoo et al, 2020;Adaloglou et al, 2020). Meanwhile, signed languages have remained relatively overlooked in NLP literature (Figure 1).…”

Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives

“…Sign Language Production Computational sign language research has been prominent for the last 30 years [3,57,63]. Previous research has focused on isolated sign recognition [1,28,45], Continuous Sign Language Recognition (CSLR) [7,17,37,36] and, more recently, the task of Sign Language Translation (SLT) [8,9,35,44]. Camgoz et al [10] proposed a jointly trained CSLR and SLT system, showing a performance increase for both tasks.…”

Aligning Subtitles in Sign Language Videos

“…For a recent comprehensive survey about sign language recognition and translation, see [33]. Here, we review relevant works on temporal localisation at the levels of individual signs and sequences, in addition to more general temporal alignment methods from the literature.…”