Sign Language Recognition Using Graph and General Deep Neural Network Based on Large Scale Dataset

Miah, Abu Saleh Musa; Hasan, Md. Al Mehedi; Nishimura, Satoshi; Shin, Jungpil

doi:10.1109/access.2024.3372425

Cited by 8 publications

(1 citation statement)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers have utilized various datasets, tools and techniques to make a system for AD classification. Recently, researchers have focused on two main approaches for training and classifying models, conventional learning and deep learning, as they proved excellent in other domains [ 28 , 29 , 30 ]. Recent studies like [ 24 , 31 , 32 , 33 , 34 ] used conventional learning for AD classification, while studies like [ 5 , 35 , 36 , 37 , 38 ] employed deep learning techniques.…”

Section: Related Workmentioning

confidence: 99%

Residual-Based Multi-Stage Deep Learning Framework for Computer-Aided Alzheimer’s Disease Detection

Hassan,

Musa Miah,

Shin

2024

J. Imaging

Self Cite

View full text Add to dashboard Cite

Alzheimer’s Disease (AD) poses a significant health risk globally, particularly among the elderly population. Recent studies underscore its prevalence, with over 50% of elderly Japanese facing a lifetime risk of dementia, primarily attributed to AD. As the most prevalent form of dementia, AD gradually erodes brain cells, leading to severe neurological decline. In this scenario, it is important to develop an automatic AD-detection system, and many researchers have been working to develop an AD-detection system by taking advantage of the advancement of deep learning (DL) techniques, which have shown promising results in various domains, including medical image analysis. However, existing approaches for AD detection often suffer from limited performance due to the complexities associated with training hierarchical convolutional neural networks (CNNs). In this paper, we introduce a novel multi-stage deep neural network architecture based on residual functions to address the limitations of existing AD-detection approaches. Inspired by the success of residual networks (ResNets) in image-classification tasks, our proposed system comprises five stages, each explicitly formulated to enhance feature effectiveness while maintaining model depth. Following feature extraction, a deep learning-based feature-selection module is applied to mitigate overfitting, incorporating batch normalization, dropout and fully connected layers. Subsequently, machine learning (ML)-based classification algorithms, including Support Vector Machines (SVM), Random Forest (RF) and SoftMax, are employed for classification tasks. Comprehensive evaluations conducted on three benchmark datasets, namely ADNI1: Complete 1Yr 1.5T, MIRAID and OASIS Kaggle, demonstrate the efficacy of our proposed model. Impressively, our model achieves accuracy rates of 99.47%, 99.10% and 99.70% for ADNI1: Complete 1Yr 1.5T, MIRAID and OASIS datasets, respectively, outperforming existing systems in binary class problems. Our proposed model represents a significant advancement in the AD-analysis domain.

show abstract

Section: Related Workmentioning

confidence: 99%

Residual-Based Multi-Stage Deep Learning Framework for Computer-Aided Alzheimer’s Disease Detection

Hassan,

Musa Miah,

Shin

2024

J. Imaging

Self Cite

View full text Add to dashboard Cite

show abstract

Spatial–temporal attention with graph and general neural network-based sign language recognition

Miah,

Hasan,

Okuyama

et al. 2024

Pattern Anal Applic

Self Cite

View full text Add to dashboard Cite

Automatic Sign Language Recognition (SLR) stands as a vital aspect within the realms of Human-Computer Interaction (HCI) and computer vision, facilitating the conversion of hand signs-utilized by individuals with significant hearing and speech impairments-into equivalent text or voice. Researchers have recently used hand skeleton joint information instead of the image pixel due to light illumination and complex background-bound problems. However, besides the hand information, body motion and facial gestures play an essential role in expressing sign language emotion. Also, a few researchers have been working to develop an SLR system by taking a multi-gesture dataset, but their performance accuracy and time complexity are not sufficient. In light of these limitations, we introduce a Spatial and Temporal Attention Model amalgamated with a General Neural Network designed for the SLR system. The main idea of our architecture is first to construct a fully connected graph to project the skeleton information. We employ self-attention mechanisms to extract insights from node Article Title and edge features across spatial and temporal domains. Our architecture bifurcates into three branches: a graph-based spatial branch, a graph-based temporal branch, and a general neural network branch, which collectively synergize to contribute to the final feature integration. Specifically, the spatial branch discerns spatial dependencies, while the temporal branch amplifies temporal dependencies embedded within the sequential hand skeleton data. Further, the general neural network branch enhances the architecture's generalization capabilities, bolstering its robustness. In our evaluation, utilizing the Mexican Sign Language(MSL), Pakistani Sign Langauge(PSL) datasets and American Sign Langauge Large Video Dataset(ASSLVD) -which comprises 3D joint coordinates for face, body, and hands that conducted experiments on individual gestures and their combinations. Impressively, our model demonstrated notable efficacy, achieving an accuracy rate of 99.96% for the MSL dataset, 92.00% for PSL, and 26.00% for the ASLLVD dataset, which includes 2745 classes. These exemplary performance metrics, coupled with the model's computationally efficient profile, underscore its preeminence compared to contemporaneous methodologies in the field.

show abstract

A multi-lingual sign language recognition system using machine learning

Najib

2024

Multimed Tools Appl

View full text Add to dashboard Cite

Recently, automatic sign language recognition field gets a great attention. Machine learning algorithms are mainly used to recognize hand gestures. Most of recent studies train their machine learning model using a specific sign language of a specific country such as the American Sign Language. In this paper, we propose a multi-lingual sign language system based machine learning that is called Multi-lingual Sign Languages Interpreter (MSLI) system. MSLI trains a machine learning model based on hand signs of multiple languages. It can detect the language of the input signs and their labels. In a case of input testing signs with the same language, the proposed system can provide two-steps recognition, where it only detects the language of the first sign, and then the rest signs are tested according to the recognized language. Also, MSLI can provide separate classification of signs per each language. Experiments were performed using 11 datasets with different languages. Separate and combined classification was performed on the input data. Experimental results show the accuracy of the proposed system. Training accuracy of the proposed system over most of the used separate different sign language datasets is approximately ranged from 90 to 100%. Also, most classification accuracy results of the test data of the separate datasets exceeded 90%. The combined classification of proposed MSLI archived training accuracy of 95.87% and testing accuracy of 92.33%.

show abstract

Sign Language Recognition Using Graph and General Deep Neural Network Based on Large Scale Dataset

Cited by 8 publications

References 43 publications

Residual-Based Multi-Stage Deep Learning Framework for Computer-Aided Alzheimer’s Disease Detection

Residual-Based Multi-Stage Deep Learning Framework for Computer-Aided Alzheimer’s Disease Detection

Spatial–temporal attention with graph and general neural network-based sign language recognition

A multi-lingual sign language recognition system using machine learning

Contact Info

Product

Resources

About