Key Frame Detection Algorithm based on Dynamic Sign Language Video for the Non Specific Population

Abstract

The current recognition algorithms of sign language, or can only identify static gestures, or need data gloves, position sensor and other additional auxiliary equipments, which are only used for laboratory research and some special occasions. Therefore, they are not conducive to the promotion of widely use. A new idea of sign language recognition based on key frames is presented in this paper. The dynamic sign language can be looked on as a series of static gestures, which can be called the key frames. Through the key frame sequence detection and identification, the sign language can be rapidly recognized. So an algorithm of key frame detection especially for the dynamic sign language is proposed. This adaptive method uses image difference and classification theory in pattern recognition to extract key frames from video, and in addition to PC machines, the entire process requires only a camera, which is very easy to use. Experiments show that the key frames obtained by this way have good stability and accuracy, thus the real-time recognition of dynamic sign language can be realized.

Real-Time Dynamic Sign Language Recognition Based on Hierarchical Matching Strategy

“…Since each key frame contains some semantics, in order to ensure not missing any key frames, the number of intervals should be 1.5 to 2 times of the possible maximum amount of the key frames. Through the study of Chinese sign language, we find that as for the vast majority of sign language words, the number of their key frames is not more than 5 [10]. As shown in Figure 2 (b), the density curve in Figure 2 (a) is evenly divided into 7 intervals, and thus 6 density extreme points are obtained.…”

Recognition of static gestures using correlation and cross-correlation

“…New methods to compare the density and diversity of image datasets and show that places are as dense as other scene datasets and has more diversity. Shurong et al (2015) have proposed that the dynamic sign language can be described by a sequence of key frames and then recognized by these key frames. This method focuses on the key frame extraction and can minimize the limitation to the users and the requests of equipment, which makes the interaction between a human and computer more natural and realizes the comprehensive application of sign language recognition.…”

Keyframe Extraction Algorithm for Continuous Sign-Language Videos Using Angular Displacement and Sequence Check Metrics

“…Te various keyframe extraction paradigms utilized in video-related computer vision tasks are cluster methods [9,10], motion energy-based methods [11], sequence methods [12][13][14][15][16], and machine learning methods [17,18]. Diferent sequential approaches and machine learning methods are the most acceptable techniques used in keyframe extraction from continuous sign-language videos.…”