SignSpeaker

Hou, Jiahui; Li, Xiangyang; Zhu, Peide; Wang, Zefan; Wang, Yu; Qian, Jianwei; Yang, Panlong

doi:10.1145/3300061.3300117

Cited by 65 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Combined with a Bi-LSTM, MyoSign achieves a classification accuracy of 93.7% for 100 different ASL words. More recently, SignSpeaker [19] has been proposed to capture hand gestures using a smartwatch platform for sign language translation. It captures IMU sensor signals from the users' smartwatch and analyzes the data using an LSTM model.…”

Section: Related Workmentioning

confidence: 99%

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

Park

Lee

2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

In this work we present SUGO, a depth video-based system for translating sign language to text using a smartphone's front camera. While exploiting depth-only videos offer benefits such as being less privacy-invasive compared to using RGB videos, it introduces new challenges which include dealing with low video resolutions and the sensors' sensitiveness towards user motion. We overcome these challenges by diversifying our sign language video dataset to be robust to various usage scenarios via data augmentation and design a set of schemes to emphasize human gestures from the input images for effective sign detection. The inference engine of SUGO is based on a 3-dimensional convolutional neural network (3DCNN) to classify a sequence of video frames as a pre-trained word. Furthermore, the overall operations are designed to be light-weight so that sign language translation takes place in real-time using only the resources available on a smartphone, with no help from cloud servers nor external sensing components. Specifically, to train and test SUGO, we collect sign language data from 20 individuals for 50 Korean Sign Language words, summing up to a dataset of ~5,000 sign gestures and collect additional in-the-wild data to evaluate the performance of SUGO in real-world usage scenarios with different lighting conditions and daily activities. Comprehensively, our extensive evaluations show that SUGO can properly classify sign words with an accuracy of up to 91% and also suggest that the system is suitable (in terms of resource usage, latency, and environmental robustness) to enable a fully mobile solution for sign language translation.

show abstract

Section: Related Workmentioning

confidence: 99%

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

Park

Lee

2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

show abstract

“…The wearable end-to-end gadgets and sensors t for sign language translation (Fang et al, 2017), (Hou et al, 2019)…”

Section: Sign Language-to-text / Language Translationmentioning

confidence: 99%

“…It is used by (Fang et al, 2017), (Jin et al, 2021 (Wang et al, 2018), (Cihan Camgoz et al, 2017), and (Hou et al, 2019). CTC is proposed by Graves et al, one of the most often used methods for developing a sequence-to-sequence model [12] and compares between speech and optical character recognition tasks (Jin et al, 2021).…”

Section: Connectionist Temporal Classification (Ctc) -mentioning

confidence: 99%

“…To translate a sign language into a text it must be recognized first. CTC has been recognized as a critical technique to separate words (Hou et al, 2019) to be able to build a sentence-level structure of translation. To be able to recognize and accurately translate a sentencelevel structure of text, it is necessary to use Bidirectional Long Short-Term Memory (BiLSTM) (Wang et al, 2018).…”

Section: Proposed Sign Language Translation Into Text Systemmentioning

confidence: 99%

“…The study is about sign language translation into text; this technology is for those who are experiencing difficulty in communicating i.e., those who have speech disorders, hearing impairment, and are deaf (Fang et al, 2017) (Ferreira et al, 2019) (Sridhar et al, 2020) (Alam et al, 2021) (Orbay and Akarun, 2020) (Shurid et al, 2020) (Hou et al, 2019)]. Sign language translation into text will help them communicate with other people as they can express their feelings thoroughly without the concern of misunderstanding them, or not understanding them at all.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Using Deep Learning in Sign Language Translation to Text

Proceedings of the International Conference on Industrial Engineering and Operations Management

View full text Add to dashboard Cite

This study is about sign language translation into text; this technology is for those experiencing difficulty in communicating, i.e., those who have speech disorders, hearing impairment, and are deaf. Sign language translation into text is done through a process when one uses sign language that could be translated into text. Deep learning is a machine learning and artificial intelligence (AI) technique that mimics how humans acquire knowledge; the complexity and abstraction of deep learning algorithms are stacked in a hierarchy. The researchers utilized Convolutional Neural Networks (CNN), Connectionist Temporal Classification (CTC), and Deep Belief Network (DBN) to collect and gather the results of their system through different methods of translating sign language into text. Through the systematic literature review of various published papers related to sign language translation using deep learning techniques, this paper discusses and systematically analyzed various studies and come up with a proposed model in translating sign language into text.

show abstract

An approach based on 1D fully convolutional network for continuous sign language recognition and labeling

Wang

Liu

et al. 2022

Neural Comput & Applic

View full text Add to dashboard Cite

SignSpeaker

Cited by 65 publications

References 24 publications

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

Using Deep Learning in Sign Language Translation to Text

An approach based on 1D fully convolutional network for continuous sign language recognition and labeling

Contact Info

Product

Resources

About