KSL-Guide: A Large-scale Korean Sign Language Dataset Including Interrogative Sentences for Guiding the Deaf and Hard-of-Hearing

Ham, Sunny; Park, Kibaek; Jang, YeongJun; Oh, Young‐Taek; Yun, Seokmin; Yoon, Sukwon; Kim, Chang Jo; Park, Han-Mu; Kweon, In So

doi:10.1109/fg52635.2021.9667011

Cited by 7 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, we evaluate our model on a newly published, large-scale Korean sign language dataset, namely the Korean Sign Language Guide Dataset (KSL-Guide) [29]. KSL-Guide is a comprehensive dataset of sign language sentences, words, and fingerspelled words and numbers used in transportation and navigation-related dialogues.…”

Section: Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Attentional bias for hands: Cascade dual‐decoder transformer for sign language production

Ma,

Jin,

Wang

et al. 2024

IET Computer Vision

View full text Add to dashboard Cite

Sign Language Production (SLP) refers to the task of translating textural forms of spoken language into corresponding sign language expressions. Sign languages convey meaning by means of multiple asynchronous articulators, including manual and non‐manual information channels. Recent deep learning‐based SLP models directly generate the full‐articulatory sign sequence from the text input in an end‐to‐end manner. However, these models largely down weight the importance of subtle differences in the manual articulation due to the effect of regression to the mean. To explore these neglected aspects, an efficient cascade dual‐decoder Transformer (CasDual‐Transformer) for SLP is proposed to learn, successively, two mappings SLPhand: Text → Hand pose and SLPsign: Text → Sign pose, utilising an attention‐based alignment module that fuses the hand and sign features from previous time steps to predict more expressive sign pose at the current time step. In addition, to provide more efficacious guidance, a novel spatio‐temporal loss to penalise shape dissimilarity and temporal distortions of produced sequences is introduced. Experimental studies are performed on two benchmark sign language datasets from distinct cultures to verify the performance of the proposed model. Both quantitative and qualitative results show that the authors’ model demonstrates competitive performance compared to state‐of‐the‐art models, and in some cases, achieves considerable improvements over them.

show abstract

Section: Datasetsmentioning

confidence: 99%

“…The data that supports the findings of this study are available in refs. [28,29]. These data were derived from the following resources available in the public domain: [PHOENIX14T: https://www-i6.informatik.rwth-aachen.de/~koller/RWTH-PHOENIX-2014-T/ and KSL-Guide: https://github.com/ ChelseaGH/KSL-Guide].…”

Section: Conflict Of Interest Statementmentioning

confidence: 99%

Attentional bias for hands: Cascade dual‐decoder transformer for sign language production

Ma,

Jin,

Wang

et al. 2024

IET Computer Vision

View full text Add to dashboard Cite

show abstract

“…Communication is an unavoidable necessity for individuals who are deaf or have hearing difficulties (DHH) in order to convey their thoughts, demands, and fundamental necessities to others. Globally, the DHH community encompasses a staggering population of 450 million individuals who rely on sign language as their primary means of communication [1]- [4]. According to the community of individuals with hearing impairments (DHH), sign language is a complex communication system that exhibits notable distinctions from conventional alphabets, words, and sentences.…”

Section: Introductionmentioning

confidence: 99%

“…This also requires effective feature extraction and classification algorithms for successful operation. To address this issue, some researchers employed a vision-based Korean Sign Language word recognition system using ANN [7], CNN [3], [4], Transformer [7], and Graph Convolutional Network (GCN) [8]. However, all existing vision-based KSL systems are designed exclusively for sign word recognition, and no research work has been found for KSL alphabet recognition.…”

Section: Introductionmentioning

confidence: 99%

Korean Sign Language Alphabet Recognition Through the Integration of Handcrafted and Deep Learning-Based Two-Stream Feature Extraction Approach

Shin,

Miah,

Akiba

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Recognizing sign language plays a crucial role in improving communication accessibility for the Deaf and hard-of-hearing communities. In Korea, many individuals facing hearing and speech challenges depend on Korean Sign Language (KSL) as their primary means of communication. Many researchers have been working to develop a sign language recognition system for other sign languages, but little research has been done for KSL alphabet recognition. However, existing KSL recognition systems have faced significant performance limitations due to the ineffectiveness of the features. To address these issues, we introduce an innovative KSL recognition system employing a strategic fusion approach. In this study, we combined joint skeleton-based handcrafted features and pixel-based resnet101 transfer learning features to overcome the limitations of traditional systems. Our proposed system consists of two distinct streams: the first stream extracts essential handcrafted features, placing emphasis on capturing hand orientation information within KSL gestures. In the second stream, concurrently, we employed a deep learning-based resnet101 module stream to capture hierarchical representations of the KSL alphabet sign. By combining essential information from the first stream with the hierarchical features from the second stream, we generate multiple levels of fused features with the goal of forming a comprehensive representation of KSL gestures. Finally, we fed the concatenated feature into the deep learning-based classification module for the classification. We conducted extensive experiments with the newly created KSL alphabet dataset, the existing KSL digit and the existing ArSL and ASL benchmark datasets. Our proposed model undeniably shows that our fusion approach substantially improves high-performance accuracy in both cases, which proves the system's superiority.INDEX TERMS Korean sign language (KSL), hand gesture recognition, geometric feature, distance feature, angle feature, ResNet.

show abstract

“…Most publicly available CSLR benchmarks are curated from either studio or TV broadcasts, where background images are fixed and monochromatic [19,28,35]. In a deployment scenario, these backgrounds are dissimilar to situations where real world communications occur, (b) Both Baseline (ResNet18 [21] + LSTM) and VAC [41] severely degrade when tested on Scene-PHOENIX.…”

Section: Introductionmentioning

confidence: 99%

Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition

Jang¹,

Oh²,

Cho³

et al. 2022

Preprint

View full text Add to dashboard Cite

The goal of this work is background-robust continuous sign language recognition. Most existing Continuous Sign Language Recognition (CSLR) benchmarks have fixed backgrounds and are filmed in studios with a static monochromatic background. However, signing is not limited only to studios in the real world. In order to analyze the robustness of CSLR models under background shifts, we first evaluate existing state-ofthe-art CSLR models on diverse backgrounds. To synthesize the sign videos with a variety of backgrounds, we propose a pipeline to automatically generate a benchmark dataset utilizing existing CSLR benchmarks. Our newly constructed benchmark dataset consists of diverse scenes to simulate a real-world environment. We observe even the most recent CSLR method cannot recognize glosses well on our new dataset with changed backgrounds. In this regard, we also propose a simple yet effective training scheme including (1) background randomization and (2) feature disentanglement for CSLR models. The experimental results on our dataset demonstrate that our method generalizes well to other unseen background data with minimal additional training images. Our dataset is available here.

show abstract

KSL-Guide: A Large-scale Korean Sign Language Dataset Including Interrogative Sentences for Guiding the Deaf and Hard-of-Hearing

Cited by 7 publications

References 17 publications

Attentional bias for hands: Cascade dual‐decoder transformer for sign language production

Attentional bias for hands: Cascade dual‐decoder transformer for sign language production

Korean Sign Language Alphabet Recognition Through the Integration of Handcrafted and Deep Learning-Based Two-Stream Feature Extraction Approach

Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition

Contact Info

Product

Resources

About