Sign-Language Recognition Through Gesture &amp; Movement Analysis (SIGMA)

Ong, Clement; Lim, Ian; Lu, Joshua; Ng, Claudine; Ong, Thomas

doi:10.1007/978-3-319-76947-9_17

Cited by 16 publications

(7 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, Ong et al [25] proposed a SIGMA system that applied vision-(using webcam) and sensor-based approaches (glove with nine resistive flex sensors which are placed on thumb (one sensor) and other fingers (two sensors each) along with 6-DOF IMUs) to recognize 26 letters, 10 digits, and 30 healthcare-related words from the Filipino Sign Language, e.g., "cough," "doctor," "physical exam," "temperature," and "allergy," with a mean accuracy rate of 71.8% for letter and digit recognition and 80.6% for word recognition using the Viterbi algorithm. In general, both visionand sensor-based approaches have their own merits and limitations.…”

Section: Related Workmentioning

confidence: 99%

Sensor Fusion of Motion-Based Sign Language Interpretation with Deep Learning

Lee

Chong

Chung

2020

Sensors

View full text Add to dashboard Cite

Sign language was designed to allow hearing-impaired people to interact with others. Nonetheless, knowledge of sign language is uncommon in society, which leads to a communication barrier with the hearing-impaired community. Many studies of sign language recognition utilizing computer vision (CV) have been conducted worldwide to reduce such barriers. However, this approach is restricted by the visual angle and highly affected by environmental factors. In addition, CV usually involves the use of machine learning, which requires collaboration of a team of experts and utilization of high-cost hardware utilities; this increases the application cost in real-world situations. Thus, this study aims to design and implement a smart wearable American Sign Language (ASL) interpretation system using deep learning, which applies sensor fusion that “fuses” six inertial measurement units (IMUs). The IMUs are attached to all fingertips and the back of the hand to recognize sign language gestures; thus, the proposed method is not restricted by the field of view. The study reveals that this model achieves an average recognition rate of 99.81% for dynamic ASL gestures. Moreover, the proposed ASL recognition system can be further integrated with ICT and IoT technology to provide a feasible solution to assist hearing-impaired people in communicating with others and improve their quality of life.

show abstract

Section: Related Workmentioning

confidence: 99%

Sensor Fusion of Motion-Based Sign Language Interpretation with Deep Learning

Lee

Chong

Chung

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…The number of researches on recognition of medical and healthcare signs is a handful. Ong et al [25] proposed a method that combines data glove with image processing to recognize ASL medical terms. Keskin et al [26] and Vidalón et al [27] proposed Hidden Markov Model (HMM) based recognition system for medical terms of Turkish and Brazilian sign language respectively.…”

Section: Sn Computer Sciencementioning

confidence: 99%

Design and Development of a Humanoid Robot for Sign Language Interpretation

et al. 2021

View full text Add to dashboard Cite

Purpose With an indisputable complexity of communication for hearing and speaking impaired people, most sign language recognition systems utilize virtual reality or onscreen robots. This paper presents the design and development of a special and low-cost humanoid robot that can perform as a sign language interpreter. To the best of our knowledge, this is the first endeavor to fabricate a humanoid robot for Bangla sign language (BdSL) and Medical signs interpretation. Methods Considering the plethora of design criteria and balancing between rigidity and flexibility 3D models of the robotics parts are designed and 3D printed ensuring cost efficiency. With the help of modern fabrication technology, the robot is developed and assembled with proper actuators and circuitry. An image dataset is built comprising 950 images for BdSL recognition and made publicly available. We utilized the Recurrent neural network (RNN) and Convolutional neural network (CNN) for deep learning model establishment and feature extraction from video and image data. Results The developed humanoid robot has 43 Degrees of freedom (DoF) which includes two 15 DoF hands. It can imitate 16 BdSL alphabets in sign language, can capture a video or image input in real-time from the user, and recognize 10 medical signs and 38 alphabets of BdSL. The learning model for video-based medical sign recognition achieved 87.5% test accuracy. Image-based Bangla sign language recognition achieved an overall test accuracy of 98.19% in our dataset and 93.8% in another available dataset. Conclusion Compared to the state-of-the-art robotic systems for sign language interpretation, our approach has achieved higher kinematic characteristics, remarkable results in sign recognition, and impressive competency in sign imitation; all at almost 10 times lower cost than the state-of-the-art systems. The results are evidence that our approach is efficient and suitable in helping hearing and speaking impaired people. Moreover, this work initiates a research scope that can be further extended for creating equal opportunities for the hearing and speaking impaired community.

show abstract

“…Static one is simpler to perceive since it involves extraction of information from still picture alone is required to recognise the same. Tremendous methods are implemented in current research to handle two‐dimensional (2D) motion recognition, including the introduction histogram, concealed Markov model, particle filtering, support vector machine (SVM) and so on [17, 18]. The vast majority of these methodologies require preprocessing the input gesture image to extract feature.…”

Section: Introductionmentioning

confidence: 99%