Vision-based hand gesture recognition of alphabets, numbers, arithmetic operators and ASCII characters in order to develop a virtual text-entry interface system

Misra, Songhita; Singha, Joyeeta; Laskar, Rabul Hussain

doi:10.1007/s00521-017-2838-6

Cited by 44 publications

(48 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main reason is that there are too many noise signals in the sample, which greatly increases the difficulty of network identification. A more skillful training strategy is needed to achieve a higher recognition rate [49][50][51]. First of all, four different training methods are used to train the DBNs network with two hidden layers, and the bottom unit of DBNs uses GB_RBM, as shown in Fig.…”

Section: Complex Background Of Gesture Recognition Results and Analysismentioning

confidence: 99%

Jointly network: a network based on CNN and RBM for gesture recognition

Cheng

Sun

et al. 2018

Neural Comput & Applic

View full text Add to dashboard Cite

Hand belongs to non-rigid objects and is rich in variety, making gesture recognition more difficult. The essence of dynamic gesture recognition is the classification and recognition of single-frame still images. Therefore, this paper mainly focuses on static gesture recognition. At present, there are some problems in gesture recognition, such as accuracy, real-time or poor robustness. To solve the above problems, in this paper, the Kinect sensor is used to obtain the color and depth gesture samples, and the gesture samples are processed. On this basis, a jointly network of CNN and RBM is proposed for gesture recognition. It mainly uses superposed network of multiple RBMs to carry out unsupervised feature extraction and combined with supervised feature extraction of CNN. Finally, these two features are combined to classify them. The simulation results show that the proposed jointly network has a better performance in identifying simple background gesture samples, and the recognition capability of gesture samples in complex background needs to be improved.

show abstract

Section: Complex Background Of Gesture Recognition Results and Analysismentioning

confidence: 99%

Jointly network: a network based on CNN and RBM for gesture recognition

Cheng

Sun

et al. 2018

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…Therefore, the value of λ is experimentally chosen as 5, to give a trade-off between accuracy and speed. Also, experiments show that the tolerance kept at 0.4 gives fairly a good (Misra et al, 2017) at a significantly lower computational cost. Visualizations for successive iterations involved in smoothing of an initially distorted character trajectory is shown in Figure 5.…”

Section: Trajectory Smoothingmentioning

confidence: 96%

Fingertip detection and tracking for recognition of air-writing in videos

Mukherjee

Sekh

Dogra

et al. 2019

Expert Systems with Applications

View full text Add to dashboard Cite

Air-writing is the process of writing characters or words in free space using finger or hand movements without the aid of any hand-held device. In this work, we address the problem of mid-air finger writing using web-cam video as input. In spite of recent advances in object detection and tracking, accurate and robust detection and tracking of the fingertip remains a challenging task, primarily due to small dimension of the fingertip. Moreover, the initialization and termination of mid-air finger writing is also challenging due to the absence of any standard delimiting criterion. To solve these problems, we propose a new writing hand pose detection algorithm for initialization of air-writing using the Faster R-CNN framework for accurate hand detection followed by hand segmentation and finally counting the number of raised fingers based on geometrical properties of the hand. Further, we propose a robust fingertip detection and tracking approach using a new signature function called distance-weighted curvature entropy. Finally, a fingertip velocity-based termination criterion is used as a delimiter to mark the completion of the air-writing gesture. Experiments show the superiority of the proposed fingertip detection and tracking algorithm over state-of-the-art approaches giving a mean precision of 73.1 % while achieving real-time performance at 18.5 fps, a condition which is of vital importance to air-writing. Character recognition experiments give a mean accuracy of 96.11 % using the proposed air-writing system, a result which is comparable to that of existing handwritten character recognition systems. IntroductionWith the emergence of virtual and augmented reality, the need for the development of natural human-computer interaction (HCI) systems to replace the traditional HCI approaches is increasing rapidly. In particular, interfaces incorporating hand gesturebased interaction have gained popularity in many fields of application viz. automotive interfaces (Ohn-Bar and Trivedi, 2014), human activity recognition (Rohrbach et al., 2016) and several state-of-the-art hand gesture recognition approaches have been developed (Molchanov et al., 2015;Rautaray and Agrawal, 2015). However, hand motion gestures as such are not sufficient to input text. This necessitates the need for the development of touch-less air-writing systems which may replace touch and electromechanical input panels leading to a more natural human-computer interaction (HCI) approach.A vision-based system for the recognition of mid-air finger-writing trajectories is not a new problem and substantial work has been done in the past two decades. One of the early works by Oka et al. (Oka et al., 2002) used a sophisticated device with an infrared and color sensor for fingertip tracking and recognition of simple geometric shapes trajectories. In (Amma et al., 2012), inertial sensors attached to a glove were used for continuous spotting and recognition of air-writing. Recently, Misra et al. (Misra et al., 2017) have developed a hand gesture recognition framewor...

show abstract

“…The focus of this method is mainly on the captured image of gesture; and the extraction of the main features for the recognition process (Kour and Mathew, 2017). The geometrical shape of trajectory and its gesture drawing speed are used to extract different features, which are then used by various pattern recognition algorithms to classify the different gestures (Misra et al, 2017).…”

Section: *Corresponding Authormentioning

confidence: 99%

Template Matching Based Sign Language Recognition System for Android Devices

2020

View full text Add to dashboard Cite

An android based sign language recognition system for selected English vocabularies was developed with the explicit objective to examine the specific characteristics that are responsible for gestures recognition. Also, a recognition model for the process was designed, implemented, and evaluated on 230 samples of hand gestures. The collected samples were pre-processed and rescaled from 3024 ×4032 pixels to 245 ×350 pixels. The samples were examined for the specific characteristics using Oriented FAST and Rotated BRIEF, and the Principal Component Analysis used for feature extraction. The model was implemented in Android Studio using the template matching algorithm as its classifier. The performance of the system was evaluated using precision, recall, and accuracy as metrics. It was observed that the system obtained an average classification rate of 87%, an average precision value of 88% and 91% for the average recall rate on the test data of hand gestures. The study, therefore, has successfully classified hand gestures for selected English vocabularies. The developed system will enhance the communication skills between hearing and hearing-impaired people, and also aid their teaching and learning processes. Future work include exploring state-of-the-art machining learning techniques such Generative Adversarial Networks (GANs) for large dataset to improve the accuracy of results. Keywords— Feature extraction; Gestures Recognition; Sign Language; Vocabulary, Android device.

show abstract

Vision-based hand gesture recognition of alphabets, numbers, arithmetic operators and ASCII characters in order to develop a virtual text-entry interface system

Cited by 44 publications

References 23 publications

Jointly network: a network based on CNN and RBM for gesture recognition

Jointly network: a network based on CNN and RBM for gesture recognition

Fingertip detection and tracking for recognition of air-writing in videos

Template Matching Based Sign Language Recognition System for Android Devices

Contact Info

Product

Resources

About