Sign Language Recognition using Sequential Pattern Trees

Ong, Eng-Jon; Cooper, Helen; Pugeault, Nicolas; Bowden, Richard

doi:10.1109/cvpr.2012.6247928

Cited by 22 publications

(8 citation statements)

References 9 publications

(10 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ong et al presented a multi-class sign language classification model that utilizes sequential pattern trees, specifically the SP-tree boosting algorithm. The primary focus of their approach was the extraction of hand trajectory features from the image subunit [17]. To assess the efficacy of their model, the researchers utilized Greek sign language (GSL), which demonstrated a notable accuracy rate of 93.00%.…”

Section: Related Workmentioning

confidence: 99%

Korean Sign Language Alphabet Recognition Through the Integration of Handcrafted and Deep Learning-Based Two-Stream Feature Extraction Approach

Shin,

Miah,

Akiba

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Recognizing sign language plays a crucial role in improving communication accessibility for the Deaf and hard-of-hearing communities. In Korea, many individuals facing hearing and speech challenges depend on Korean Sign Language (KSL) as their primary means of communication. Many researchers have been working to develop a sign language recognition system for other sign languages, but little research has been done for KSL alphabet recognition. However, existing KSL recognition systems have faced significant performance limitations due to the ineffectiveness of the features. To address these issues, we introduce an innovative KSL recognition system employing a strategic fusion approach. In this study, we combined joint skeleton-based handcrafted features and pixel-based resnet101 transfer learning features to overcome the limitations of traditional systems. Our proposed system consists of two distinct streams: the first stream extracts essential handcrafted features, placing emphasis on capturing hand orientation information within KSL gestures. In the second stream, concurrently, we employed a deep learning-based resnet101 module stream to capture hierarchical representations of the KSL alphabet sign. By combining essential information from the first stream with the hierarchical features from the second stream, we generate multiple levels of fused features with the goal of forming a comprehensive representation of KSL gestures. Finally, we fed the concatenated feature into the deep learning-based classification module for the classification. We conducted extensive experiments with the newly created KSL alphabet dataset, the existing KSL digit and the existing ArSL and ASL benchmark datasets. Our proposed model undeniably shows that our fusion approach substantially improves high-performance accuracy in both cases, which proves the system's superiority.INDEX TERMS Korean sign language (KSL), hand gesture recognition, geometric feature, distance feature, angle feature, ResNet.

show abstract

Section: Related Workmentioning

confidence: 99%

Korean Sign Language Alphabet Recognition Through the Integration of Handcrafted and Deep Learning-Based Two-Stream Feature Extraction Approach

Shin,

Miah,

Akiba

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Researchers employed various technologies to develop the hand gesture-based SLR system, specifically hand-crafted feature extraction with machine learning algorithms and deep learning algorithms [19], [32]. Many researchers extracted hand-crafted features and employed machine learning algorithms such as the Hidden Markov model (HMM), [1], Pattern Trees (SP-Tree) [33] and they reported 93.00% accuracy for Greek Sign Language (GSL) and 88.00% accuracy for German Sign Language (GSL) respectively. Sequentially, Linear Discriminant Analysis (LDA), k-nearest Neighbors (KNN), and Random Decision Forest (RDF) also proved their efficiency for various SL datasets [19].…”

Section: Related Workmentioning

confidence: 99%

Hand Gesture Recognition for Multi-Culture Sign Language Using Graph and General Deep Learning Network

Miah,

Hasan,

Tomioka

et al. 2024

IEEE Open J. Comput. Soc.

View full text Add to dashboard Cite

Hand gesture-based Sign Language Recognition (SLR) serves as a crucial communication bridge between deaf and non-deaf individuals. The absence of a universal sign language (SL) leads to diverse nationalities having various cultural SLs, such as Korean, American, and Japanese sign language. Existing SLR systems perform well for their cultural SL but may struggle with other or multi-cultural sign languages (McSL). To address these challenges, this paper introduces a novel end-to-end SLR system called GmTC, designed to translate McSL into equivalent text for enhanced understanding. Here, we employed a Graph and General deep-learning network as two stream modules to extract effective features. In the first stream, produce a graph-based feature by taking advantage of the superpixel values and the graph convolutional network (GCN), aiming to extract distance-based complex relationship features among the superpixel. In the second stream, we extracted long-range and short-range dependency features using attention-based contextual information that passes through multi-stage, multi-head self-attention (MHSA), and CNN modules. Combining these features generates final features that feed into the classification module. Extensive experiments with four culture SL datasets with high-performance accuracy compared to existing state-of-the-art models in individual domains affirming superiority and generalizability.

show abstract

“…The system obtained an accuracy of 99% with VGG19 and 97% with the CNN architecture. E. J. Ong et al [6] suggested a Sequential Pattern Tree-based (SP-Tree) multi-class classifier for German Sign Language (DGS) and Greek Sign Language (GSL) fingerspelling recognition. Their proposed SP-Tree Boosting algorithm-based recognition model performs better than the Hidden Markov Model (HMM).…”

Section: Introductionmentioning

confidence: 99%

Thai Finger-Spelling using Vision Transformer

Chaowanawatee,

Silanon,

Kliangsuwan

2023

IJACSA

View full text Add to dashboard Cite

In this paper, we present a finger-spelling recognition system that is based on Thai Sign Language (TFS) and employs a deep learning model called vision transformer. We extracted the 15 characters of the Thai alphabet from publicly available and our collected datasets to establish the recognition system. To train the learning model, we employed four EVA-02 vision transformer models, each of which showed impressive performance across different model sizes. We conducted four experiments to determine the most effective performance model. In Experiment 1, we directly trained the model to compare its performance. In Experiment 2, we used augmentation techniques to generate additional datasets. Experiment 3 utilized the Test-Time Augmentation (TTA) technique to generate test images with random variations. Lastly, in Experiment 4, we used Pseudo-Labelling (labeling labeled and unlabeled data) in each batch to train the model network. Furthermore, we developed a mobile application that collects user image data and provides helpful information related to finger-spelling, such as meanings, gestures, and usage examples.

show abstract

Sign Language Recognition using Sequential Pattern Trees

Cited by 22 publications

References 9 publications

Korean Sign Language Alphabet Recognition Through the Integration of Handcrafted and Deep Learning-Based Two-Stream Feature Extraction Approach

Korean Sign Language Alphabet Recognition Through the Integration of Handcrafted and Deep Learning-Based Two-Stream Feature Extraction Approach

Hand Gesture Recognition for Multi-Culture Sign Language Using Graph and General Deep Learning Network

Thai Finger-Spelling using Vision Transformer

Contact Info

Product

Resources

About