Deepsign: Sign Language Detection and Recognition Using Deep Learning

Kothadiya, Deep; Bhatt, Chintan; Sapariya, Krenil; Patel, Kevin; González, Ana Belén Gil; Corchado, Juan M.

doi:10.3390/electronics11111780

Cited by 64 publications

(12 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At last, comprehensive comparative results of the ASODCAE-SLR model with recent models are given in Figure 10 [ 21 ]. The figure indicated that the GRU-LSTM model has attained reduced classification results compared to existing techniques.…”

Section: Results Analysismentioning

confidence: 99%

Atom Search Optimization with Deep Learning Enabled Arabic Sign Language Recognition for Speaking and Hearing Disability Persons

et al. 2022

View full text Add to dashboard Cite

Sign language has played a crucial role in the lives of impaired people having hearing and speaking disabilities. They can send messages via hand gesture movement. Arabic Sign Language (ASL) recognition is a very difficult task because of its high complexity and the increasing intraclass similarity. Sign language may be utilized for the communication of sentences, letters, or words using diverse signs of the hands. Such communication helps to bridge the communication gap between people with hearing impairment and other people and also makes it easy for people with hearing impairment to express their opinions. Recently, a large number of studies have been ongoing in developing a system that is capable of classifying signs of dissimilar sign languages into the given class. Therefore, this study designs an atom search optimization with a deep convolutional autoencoder-enabled sign language recognition (ASODCAE-SLR) model for speaking and hearing disabled persons. The presented ASODCAE-SLR technique mainly aims to assist the communication of speaking and hearing disabled persons via the SLR process. To accomplish this, the ASODCAE-SLR technique initially pre-processes the input frames by a weighted average filtering approach. In addition, the ASODCAE-SLR technique employs a capsule network (CapsNet) feature extractor to produce a collection of feature vectors. For the recognition of sign language, the DCAE model is exploited in the study. At the final stage, the ASO algorithm is utilized as a hyperparameter optimizer which in turn increases the efficacy of the DCAE model. The experimental validation of the ASODCAE-SLR model is tested using the Arabic Sign Language dataset. The simulation analysis exhibit the enhanced performance of the ASODCAE-SLR model compared to existing models.

show abstract

Section: Results Analysismentioning

confidence: 99%

Atom Search Optimization with Deep Learning Enabled Arabic Sign Language Recognition for Speaking and Hearing Disability Persons

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The authors of Ref. [ 47 ] proposed a deep-learning-based model that detects and recognizes words from a person’s gestures. The proposed model consisted of a single layer of LSTM followed by GRU.…”

Section: Discussion and Significance Of The Proposed Workmentioning

confidence: 99%

A Sign Language Recognition System Applied to Deaf-Mute Medical Consultation

Xia

Lü

Fan

et al. 2022

Sensors

View full text Add to dashboard Cite

It is an objective reality that deaf-mute people have difficulty seeking medical treatment. Due to the lack of sign language interpreters, most hospitals in China currently do not have the ability to interpret sign language. Normal medical treatment is a luxury for deaf people. In this paper, we propose a sign language recognition system: Heart-Speaker. Heart-Speaker is applied to a deaf-mute consultation scenario. The system provides a low-cost solution for the difficult problem of treating deaf-mute patients. The doctor only needs to point the Heart-Speaker at the deaf patient and the system automatically captures the sign language movements and translates the sign language semantics. When a doctor issues a diagnosis or asks a patient a question, the system displays the corresponding sign language video and subtitles to meet the needs of two-way communication between doctors and patients. The system uses the MobileNet-YOLOv3 model to recognize sign language. It meets the needs of running on embedded terminals and provides favorable recognition accuracy. We performed experiments to verify the accuracy of the measurements. The experimental results show that the accuracy rate of Heart-Speaker in recognizing sign language can reach 90.77%.

show abstract

“…SL is also known as a visual language which is generally composed of several visual partials, such as gestures, facial expressions, head pose and body postures. Specifically, six basic parameters are listed as the basic components of SL in [51], i.e., hand shape, orientation, movement, location, mouth shape and eyebrow movements. Taking an overall perspective into account, we regard gestures, facial expressions, and head poses as the primary visual modalities in SL.…”

Section: Sign Languagementioning

confidence: 99%

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Liu

Wang

et al. 2021

Front. Neurorobot.

View full text Add to dashboard Cite

The redundant information, noise data generated in the process of single-modal feature extraction, and traditional learning algorithms are difficult to obtain ideal recognition performance. A multi-modal fusion emotion recognition method for speech expressions based on deep learning is proposed. Firstly, the corresponding feature extraction methods are set up for different single modalities. Among them, the voice uses the convolutional neural network-long and short term memory (CNN-LSTM) network, and the facial expression in the video uses the Inception-Res Net-v2 network to extract the feature data. Then, long and short term memory (LSTM) is used to capture the correlation between different modalities and within the modalities. After the feature selection process of the chi-square test, the single modalities are spliced to obtain a unified fusion feature. Finally, the fusion data features output by LSTM are used as the input of the classifier LIBSVM to realize the final emotion recognition. The experimental results show that the recognition accuracy of the proposed method on the MOSI and MELD datasets are 87.56 and 90.06%, respectively, which are better than other comparison methods. It has laid a certain theoretical foundation for the application of multimodal fusion in emotion recognition.

show abstract

Deepsign: Sign Language Detection and Recognition Using Deep Learning

Cited by 64 publications

References 12 publications

Atom Search Optimization with Deep Learning Enabled Arabic Sign Language Recognition for Speaking and Hearing Disability Persons

Atom Search Optimization with Deep Learning Enabled Arabic Sign Language Recognition for Speaking and Hearing Disability Persons

A Sign Language Recognition System Applied to Deaf-Mute Medical Consultation

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Contact Info

Product

Resources

About