Korean sign language recognition based on image and convolution neural network

Shin, Hyojoo; Kim, Woo-Je; Jang, Kyoung‐Ae

doi:10.1145/3313950.3313967

Cited by 15 publications

(10 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar to action recognition, some recent works [56,35] use CNNs to extract the holistic features from image frames and then use the extracted features for classification. Several approaches [37,36] first extract body keypoints and then concatenate their locations as a feature vector.…”

Section: Sign Language Recognition Approachesmentioning

confidence: 99%

Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

Opazo

et al. 2020

2020 IEEE Winter Conference on Applications of Computer Vision (WACV)

282

239

View full text Add to dashboard Cite

Vision-based sign language recognition aims at helping the hearing-impaired people to communicate with others. However, most existing sign language datasets are limited to a small number of words. Due to the limited vocabulary size, models learned from those datasets cannot be applied in practice. In this paper, we introduce a new largescale Word-Level American Sign Language (WLASL) video dataset, containing more than 2000 words performed by over 100 signers. This dataset will be made publicly available to the research community. To our knowledge,it is by far the largest public ASL dataset to facilitate word-level sign recognition research.Based on this new large-scale dataset, we are able to experiment several deep learning methods for word-level sign recognition and evaluate their performances in large scale scenarios. Specifically we implement and compare two different models,i.e., (i) holistic visual appearance based approach, and (ii) 2D human pose based approach. Both models are valuable baselines that will benefit the community for method benchmarking. Moreover, we also propose a novel pose-based temporal graph convolution networks (Pose-TGCN) that models spatial and temporal dependencies in human pose trajectories simultaneously, which has further boosted the performance of the pose-based method. Our results show that pose-based and appearance-based models achieve comparable performances up to 62.63% at top-10 accuracy on 2,000 words/glosses, demonstrating the validity and challenges of our dataset. We will make the large-scale dataset, as well as our baseline deep models, freely available on github.

show abstract

Section: Sign Language Recognition Approachesmentioning

confidence: 99%

Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

Opazo

et al. 2020

2020 IEEE Winter Conference on Applications of Computer Vision (WACV)

282

239

View full text Add to dashboard Cite

show abstract

“…ISLR shares a lot of features with action recognition, and consequently there are several works using CNNs for feature extraction and classification [ 32 , 33 , 34 , 35 ]. Recent work has also relied on employing 3D-CNNs [ 36 , 37 ] to capture spatiotemporal information in an ensemble way.…”

Section: Related Workmentioning

confidence: 99%

Synthetic Corpus Generation for Deep Learning-Based Translation of Spanish Sign Language

Perea-Trigo,

Botella-López,

Martínez-del-Amor

et al. 2024

Sensors

View full text Add to dashboard Cite

Sign language serves as the primary mode of communication for the deaf community. With technological advancements, it is crucial to develop systems capable of enhancing communication between deaf and hearing individuals. This paper reviews recent state-of-the-art methods in sign language recognition, translation, and production. Additionally, we introduce a rule-based system, called ruLSE, for generating synthetic datasets in Spanish Sign Language. To check the usefulness of these datasets, we conduct experiments with two state-of-the-art models based on Transformers, MarianMT and Transformer-STMC. In general, we observe that the former achieves better results (+3.7 points in the BLEU-4 metric) although the latter is up to four times faster. Furthermore, the use of pre-trained word embeddings in Spanish enhances results. The rule-based system demonstrates superior performance and efficiency compared to Transformer models in Sign Language Production tasks. Lastly, we contribute to the state of the art by releasing the generated synthetic dataset in Spanish named synLSE.

show abstract

“…Apart from these, we have also studied Chinese Sign Language (Chai et al, 2013), German Sign Language (Dreuw, Deselaers, Keysers, & Ney, 2006), Korean Sign Language (Shin, Kim, & Jang, 2019), Japanese Sign Language (Sako & Kitamura, 2013), etc., to build a robust system that can recognize sign language accurately.…”

Section: Other Sign Languagesmentioning

confidence: 99%

A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier

Das

Imtiaz²,

Neom³

et al. 2023

Expert Systems with Applications

View full text Add to dashboard Cite

Korean sign language recognition based on image and convolution neural network

Cited by 15 publications

References 5 publications

Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

Synthetic Corpus Generation for Deep Learning-Based Translation of Spanish Sign Language

A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier

Contact Info

Product

Resources

About