American sign language recognition and training method with recurrent neural network

Lee, C. K. M.; Ng, Ka Man; Chen, Chun‐Hsien; Lau, Henry C. W.; Chung, Seungeun; Tsoi, Tiffany

doi:10.1016/j.eswa.2020.114403

Cited by 98 publications

(38 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LSTM is a family of RNN to handle gradient vanishing, by substituting an extended bidirectional LSTM (BiLSTM) neurons [ 27 , 54 , 55 ]. BiLSTM neuron learn long-term dependencies between sequences [ 5 , 31 , 56 ]. Single BiLSTM unit return low accuracy especially when learning complex sequences.…”

Section: Methodsmentioning

confidence: 99%

“…However, it is demonstrated that

and

should be maximized and

and

minimized to better explore performance of selected features and to determine optimal multi-stacked deep BiLSTM recognition. The following metrics are most popular for deep neural network and provide the results of comparison [ 5 ].…”

Section: Methodsmentioning

confidence: 99%

“…Among sign languages, which are normally used in deaf communication, American sign language (ASL) is one of the standard [ 1 , 2 , 3 ] and popularly used sign language across the world. ASL words are performed using single and double hands in the deaf communication, and majority of ASL words are performed using double hands, which are dominant and non-dominant hands [ 4 , 5 ]. Several single-handed words have now added a second hand in an identical or reciprocal rotation, to increase redundancy.…”

Section: Introductionmentioning

confidence: 99%

“…Research works in dynamic sign word recognition are grouped into single- and double-hand dynamic sign words. Works in single-hand [ 5 , 14 , 15 , 16 , 17 ] dynamic sign words are limited due to the fact most of the sign letters or alphabets are less complex. Double-hand sign words are commonly used in daily communications, but complex hand motion interactions are major challenge.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

American Sign Language Words Recognition of Skeletal Videos Using Processed Video Driven Multi-Stacked Deep LSTM

Abdullahi

Chamnongthai

2022

Sensors

View full text Add to dashboard Cite

Complex hand gesture interactions among dynamic sign words may lead to misclassification, which affects the recognition accuracy of the ubiquitous sign language recognition system. This paper proposes to augment the feature vector of dynamic sign words with knowledge of hand dynamics as a proxy and classify dynamic sign words using motion patterns based on the extracted feature vector. In this method, some double-hand dynamic sign words have ambiguous or similar features across a hand motion trajectory, which leads to classification errors. Thus, the similar/ambiguous hand motion trajectory is determined based on the approximation of a probability density function over a time frame. Then, the extracted features are enhanced by transformation using maximal information correlation. These enhanced features of 3D skeletal videos captured by a leap motion controller are fed as a state transition pattern to a classifier for sign word classification. To evaluate the performance of the proposed method, an experiment is performed with 10 participants on 40 double hands dynamic ASL words, which reveals 97.98% accuracy. The method is further developed on challenging ASL, SHREC, and LMDHG data sets and outperforms conventional methods by 1.47%, 1.56%, and 0.37%, respectively.

show abstract

Section: Methodsmentioning

confidence: 99%

“…However, it is demonstrated that

and

should be maximized and

and

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

American Sign Language Words Recognition of Skeletal Videos Using Processed Video Driven Multi-Stacked Deep LSTM

Abdullahi

Chamnongthai

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Recurrent Neural Networks (RNN) are network structures for which the inputs are time-series data and all the nodes are connected in a chain [73]. Unlike multi-layer perceptrons, the RNN have a sense of time and memory of earlier network states allowing them to learn sequences that vary over time [74] (see Figure 4). At present, the most commonly used RNNs are Long Short-Term Memory networks (LSTM) and Gated Recurrent Unit (GRU) networks.…”

Section: B Recurrent Neural Network Based Fault Diagnosismentioning

confidence: 99%

Deep Learning-Based Fault Diagnosis of Photovoltaic Systems: A Comprehensive Review and Enhancement Prospects

Mansouri¹,

Trabelsi

Nounou

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Photovoltaic (PV) systems are subject to failures during their operation due to the aging effects and external/environmental conditions. These faults may affect the different system components such as PV modules, connection lines, converters/inverters, which can lead to a decrease in the efficiency, performance, and further system collapse. Thus, a key factor to be taken into consideration in high-efficiency grid-connected PV systems is the fault detection and diagnosis (FDD). The performance of the FDD method depends mainly on the quality of the extracted features including real-time changes, phase changes, trend changes, and faulty modes. Thus, the data representation learning is the core stage of intelligent FDD techniques. Recently, due to the enhancement of computing capabilities, the increase of the big data use, and the development of effective algorithms, the deep learning (DL) tool has witnessed a great success in data science. Therefore, this paper proposes an extensive review on deep learning based FDD methods for PV systems. After a brief description of the DL-based strategies, techniques for diagnosing PV systems proposed in recent literature are overviewed and analyzed to point out their differences, advantages and limits. Future research directions towards the improvement of the performance of the DL-based FDD techniques are also discussed. This review paper aims to systematically present the development of DLbased FDD for PV systems and provide guidelines for future research in the field.

show abstract

Implementation of a Lightweight CNN for American Sign Language Classification

Lomas¹,

Quelal²,

Morocho-Cayamcela³

2022

Communications in Computer and Information Science

View full text Add to dashboard Cite

American sign language recognition and training method with recurrent neural network

Cited by 98 publications

References 46 publications

American Sign Language Words Recognition of Skeletal Videos Using Processed Video Driven Multi-Stacked Deep LSTM

American Sign Language Words Recognition of Skeletal Videos Using Processed Video Driven Multi-Stacked Deep LSTM

Deep Learning-Based Fault Diagnosis of Photovoltaic Systems: A Comprehensive Review and Enhancement Prospects

Implementation of a Lightweight CNN for American Sign Language Classification

Contact Info

Product

Resources

About