An Efficient Sign Language Recognition (SLR) System Using Camshift Tracker and Hidden Markov Model (HMM)

Abstract

An efficient Sign Language Recognition (SLR) system could facilitate communication with hearing impaired persons by identifying the sign gestures. Similar to regional spoken languages, different regions have developed their own sign gesture representations (for example, American Sign Language (ASL), German Sign Language (GSL), Indian Sign Language (ISL), etc.). Such variations in the hand shapes and movements add many challenges in the recognition process. The overall SLR process can be divided into a number of modules such as hand and face detection, hand tracking, features extraction and gesture recognition. In this paper, we propose a novel end-to-end SLR system from RGB video-sequences. After detecting the skin color from video frames, Camshift tracker is employed to extract the trajectories of hand motion. Next, Hidden Markov Model (HMM) based sequence classification is used to recognize the gestures. A novel approach identifying double and single hand gestures is also proposed. Furthermore new features, from skin region and hand trajectories, are proposed to improve the gesture classification performance. We tested our system on dataset proposed by American Sign Language Linguistic Research Project (ASLLRP) [25], which consists of isolated signs. The experiment results are encouraging.

[Retracted] Adaptive Gaussian Incremental Expectation Stadium Parameter Estimation Algorithm for Sports Video Analysis

“…Due to the nonrigid, nonsymmetrical, and polymorphic characteristics of human targets, coupled with the vulnerability of human targets to interference from occlusion, light changes, and complex backgrounds, human target tracking and its behavior recognition is always a pressing challenge to be solved [2]. e research on this topic can be summarized in two important processes: one is human target tracking and localization, the other is action recognition and behavior understanding, the former is the foundation, and the latter is a higher level of application.…”

Methods, Databases and Recent Advancement of Vision-Based Hand Gesture Recognition for HCI Systems: A Review

“…Because of this problem, a conventional mean-shift tracker fails to position a fast-moving object. [ 152 , 185 , 190 ] used a modified mean-shift algorithm called continuous adaptive mean-shift (CAMShift) where the window size is adjusted so as to fit the gesture area reflected by any variation in the distance between the camera and the hand. Though CAMShift performs well with objects that have a simple and consistent appearance, it is not powerful in more perplexing scenes.…”

Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues

“…In [87], a benchmark signer is selected and input from other signers is standardized based on positions of key joints. Contour extraction is used to this end as well, for example in [88], with the main focus on the areas corresponding to hands, with background removed from the image. For SLR methods that rely primarily on video for raw input, frame downsampling is frequently used to standardize the quality of various clips and reduce computational demands.…”

“…Content may change prior to final publication. [136] but this methodology could conceivably also be used with different modalities [88]. Versatility of the deep learning approach with transformer architecture is very welcome in this challenging field, since the output can be specialized through the selection of training dataset and features, as well as training hyper parameters.…”