Technological Solutions for Sign Language Recognition: A Scoping Review of Research Trends, Challenges, and Opportunities

Joksimoski, Boban; Zdravevski, Eftim; Lameski, Petre; Pires, Ivan Miguel; Melero, Francisco José; Martinez, Tomas Puebla; García, Nuno M.; Mihajlov, Martin; Chorbev, Ivan; Trajkovik, Vladimir

doi:10.1109/access.2022.3161440

Cited by 24 publications

(13 citation statements)

References 108 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Text2sign [21] HLSTM [22] ESN [23] DiffGAN [24] RG [25] SEEG [26] HA2G [27] Paul Duchnowski [28] Gérard Bailly [29] Wav2lip [30] Audio2head [31] AD-NERF [32] DiffTalk [33] Rule-based [34] LBP [35] SDF [36] PTSLP [37] Syn [38] MMFSL [39] Re-Syn [40] CMML [41] DTW [42] HMMs [43] FCN [44] RL [45] CS [1] MCCS [46] SLreview [2] UnorgSign [47] CoSreview [6] CoSreview [3] SE [48] THreview [4] THE [49] VHTHG [50] Fig. 3: Structured taxonomy of the existing BL research which includes three genres.…”

Section: Sign Language Recognitionmentioning

confidence: 99%

“…SL is categorized as a natural language commonly used in deaf communities [2]. Based on data from the World Federation of the Deaf, the worldwide population of the deaf is estimated to be around 72 million, with over 80% living in developing nations [47].…”

Section: Sign Languagementioning

confidence: 99%

“…In this survey, we primarily focus on four typical BLs and use them as examples to review and analyze the multi-modal BL recognition and generation. Figure 1 presents a simple diagram for the four types of BLs, i.e., Cued Speech (CS) [1], Sign Language (SL) [2], Co-speech (CoS) [3] and Talking Head (TH) [4]. In this field, there have been numerous previous works, which have made significant progress.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Liu

Wang

et al. 2021

Front. Neurorobot.

View full text Add to dashboard Cite

The redundant information, noise data generated in the process of single-modal feature extraction, and traditional learning algorithms are difficult to obtain ideal recognition performance. A multi-modal fusion emotion recognition method for speech expressions based on deep learning is proposed. Firstly, the corresponding feature extraction methods are set up for different single modalities. Among them, the voice uses the convolutional neural network-long and short term memory (CNN-LSTM) network, and the facial expression in the video uses the Inception-Res Net-v2 network to extract the feature data. Then, long and short term memory (LSTM) is used to capture the correlation between different modalities and within the modalities. After the feature selection process of the chi-square test, the single modalities are spliced to obtain a unified fusion feature. Finally, the fusion data features output by LSTM are used as the input of the classifier LIBSVM to realize the final emotion recognition. The experimental results show that the recognition accuracy of the proposed method on the MOSI and MELD datasets are 87.56 and 90.06%, respectively, which are better than other comparison methods. It has laid a certain theoretical foundation for the application of multimodal fusion in emotion recognition.

show abstract

Section: Sign Language Recognitionmentioning

confidence: 99%

Section: Sign Languagementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Liu

Wang

et al. 2021

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…Through using speech-to-text (STT) conversion technology, it provides a way for the hearing impaired to read a text. [1] The primary form of communication for the deaf and hard of hearing with their family members and the rest of society is sign language, which is essential in transferring meaning through a visual-manual modality. The study of sign languages has shown a new promise with the developments in computer graphics, computer vision, neural networks, and the advent of new powerful hardware.…”

Section: Introductionmentioning

confidence: 99%

Real Time Sign Language Recognition Using Computer Vision

2023

IRJMETS

View full text Add to dashboard Cite

A disability that hinders a person's capacity for verbal communication is linguistic impediment. Among the most structured languages is sign language, which would be designed to address this issue. There seems to be undeniably a requirement for a system or software program which can detect sign language gestures, allowing for communication even with individuals who do not understand sign language. A real-time system was developed with the aid of machine learning and image processing. Pre-processing images and eradicating multiple hands from the background are both done via image processing. These images, which were taken after the background was removed, were used to create data that contained the 26 English alphabets. Indeed a customized dataset and spontaneous hand gestures performed by individuals of varied complexion have been employed to evaluate the convolutional neural network that has been proposed here.

show abstract

“…A review of CSLR literature [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 ] reveals a growing body of research focused on developing robust and efficient recognition systems. However, no work related to CSLR with intonation-colored speech has been carried out for the Kazakh language.…”

Section: Introductionmentioning

confidence: 99%

Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech

Amangeldy

Ukenova

Bekmanova

et al. 2023

Sensors

View full text Add to dashboard Cite

This article is devoted to solving the problem of converting sign language into a consistent text with intonation markup for subsequent voice synthesis of sign phrases by speech with intonation. The paper proposes an improved method of continuous recognition of sign language, the results of which are transmitted to a natural language processor based on analyzers of morphology, syntax, and semantics of the Kazakh language, including morphological inflection and the construction of an intonation model of simple sentences. This approach has significant practical and social significance, as it can lead to the development of technologies that will help people with disabilities to communicate and improve their quality of life. As a result of the cross-validation of the model, we obtained an average test accuracy of 0.97 and an average val_accuracy of 0.90 for model evaluation. We also identified 20 sentence structures of the Kazakh language with their intonational model.

show abstract

Technological Solutions for Sign Language Recognition: A Scoping Review of Research Trends, Challenges, and Opportunities

Cited by 24 publications

References 108 publications

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Real Time Sign Language Recognition Using Computer Vision

Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech

Contact Info

Product

Resources

About