Hand Gesture Recognition for Multi-Culture Sign Language Using Graph and General Deep Learning Network

Abstract

Hand gesture-based Sign Language Recognition (SLR) serves as a crucial communication bridge between deaf and non-deaf individuals. The absence of a universal sign language (SL) leads to diverse nationalities having various cultural SLs, such as Korean, American, and Japanese sign language. Existing SLR systems perform well for their cultural SL but may struggle with other or multi-cultural sign languages (McSL). To address these challenges, this paper introduces a novel end-to-end SLR system called GmTC, designed to translate McSL into equivalent text for enhanced understanding. Here, we employed a Graph and General deep-learning network as two stream modules to extract effective features. In the first stream, produce a graph-based feature by taking advantage of the superpixel values and the graph convolutional network (GCN), aiming to extract distance-based complex relationship features among the superpixel. In the second stream, we extracted long-range and short-range dependency features using attention-based contextual information that passes through multi-stage, multi-head self-attention (MHSA), and CNN modules. Combining these features generates final features that feed into the classification module. Extensive experiments with four culture SL datasets with high-performance accuracy compared to existing state-of-the-art models in individual domains affirming superiority and generalizability.

Residual-Based Multi-Stage Deep Learning Framework for Computer-Aided Alzheimer’s Disease Detection

“…Recently, researchers have focused on two main approaches for training and classifying models, conventional learning and deep learning, as they proved excellent in other domains [ 28 , 29 , 30 ]. Recent studies like [ 24 , 31 , 32 , 33 , 34 ] used conventional learning for AD classification, while studies like [ 5 , 35 , 36 , 37 , 38 ] employed deep learning techniques.…”