A Comprehensive Study on Deep Learning-Based Methods for Sign Language Recognition

Adaloglou, Nikolas; Chatzis, Theocharis; Papastratis, Ilias; Stergioulas, Andreas; Papadopoulos, Georgios Th.; Zacharopoulou, Vassia; Xydopoulos, George J.; Atzakas, Klimnis; Papazachariou, Dimitris; Daras, Petros

doi:10.1109/tmm.2021.3070438

Cited by 79 publications

(38 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sincan et al in [ 16 ], captured isolated Turkish sign language glosses using Kinect sensors with a large variety of indoor and outdoor backgrounds, revealing the importance of capturing videos with various backgrounds. Adaloglou et al in [ 17 ], created a large sign language dataset with RealSense D435 sensor that records both RGB and depth information. The dataset contain continuous and isolated sign videos and is appropriate for both isolated and continuous sign language recognition tasks.…”

Section: Sign Language Capturingmentioning

confidence: 99%

“…GRSL [ 15 ] is another CSLR dataset of Greek sign language that is used in home care services, which contains multiple modalities, such as RGB, depth and skeletal joints. On the other hand, GSL [ 17 ] is a large Greek sign language dataset created to assist communication of Deaf people with public service employees. The dataset was created with a RealSense D435 sensor that records both RGB and depth information.…”

Section: Sign Language Capturingmentioning

confidence: 99%

See 1 more Smart Citation

Artificial Intelligence Technologies for Sign Language

Papastratis

Chatzikonstantinou

Konstantinidis

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

AI technologies can play an important role in breaking down the communication barriers of deaf or hearing-impaired people with other communities, contributing significantly to their social inclusion. Recent advances in both sensing technologies and AI algorithms have paved the way for the development of various applications aiming at fulfilling the needs of deaf and hearing-impaired communities. To this end, this survey aims to provide a comprehensive review of state-of-the-art methods in sign language capturing, recognition, translation and representation, pinpointing their advantages and limitations. In addition, the survey presents a number of applications, while it discusses the main challenges in the field of sign language technologies. Future research direction are also proposed in order to assist prospective researchers towards further advancing the field.

show abstract

Section: Sign Language Capturingmentioning

confidence: 99%

Section: Sign Language Capturingmentioning

confidence: 99%

Artificial Intelligence Technologies for Sign Language

Papastratis

Chatzikonstantinou

Konstantinidis

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Random sampling or discarding of frames is one of the most straightforward techniques found in literature, where approximately 20% of input is eliminated. In [89], this technique is complemented by random changes of brightness, saturation, and other image parameters. Some of the data augmentation methods used in [90] include Gaussian Noise, Just Counter, and Future Prediction.…”

Section: ) Normalization and Filteringmentioning

confidence: 99%

“…Therefore, feature extraction is another necessary element of all transformer-based neural models, where the most relevant features derived from input tokens are selected and later used for model training [85], [136]. Some of the features delineate between signs (inter-cue features), while others are useful to differentiate the particular gloss from similar ones (intra-cue features) [89], [137].…”

Section: ) Transformer-based Approachmentioning

confidence: 99%

“…Low accuracy is especially apparent with more complex tasks, and it tends to decrease as the complexity of analyzed sign language samples grows [139]. It's possible that gaps in performance are due to training samples and selected features rather than the fundamental data processing approach, but this postulation needs to be ascertained by more comprehensive testing and the possible inclusion of additional input modalities and localized sign language variations [89]. Based on the presented results, universal autonomous tools capable of continuous SLR that is signer-independent and language-independent remains a distant goal.…”

Section: ) Transformer-based Approachmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues

Al‐Qurishi¹,

Khalid²,

Souissi³

2021

IEEE Access

View full text Add to dashboard Cite

People with hearing impairments are found worldwide; therefore, the development of effective local level sign language recognition (SLR) tools is essential. We conducted a comprehensive review of automated sign language recognition based on machine/deep learning methods and techniques published between 2014 and 2021 and concluded that the current methods require conceptual classification to interpret all available data correctly. Thus, we turned our attention to elements that are common to almost all sign language recognition methodologies. This paper discusses their relative strengths and weaknesses, and we propose a general framework for researchers. This study also indicates that input modalities bear great significance in this field; it appears that recognition based on a combination of data sources, including vision-based and sensor-based channels, is superior to a unimodal analysis. In addition, recent advances have allowed researchers to move from simple recognition of sign language characters and words towards the capacity to translate continuous sign language communication with minimal delay. Many of the presented models are relatively effective for a range of tasks, but none currently possess the necessary generalization potential for commercial deployment. However, the pace of research is encouraging, and further progress is expected if specific difficulties are resolved.

show abstract