Natural human-computer interaction using static and dynamic hand gestures

Designing of touchless user interface is gaining popularity in various contexts. Using such interfaces, users can interact with electronic devices even when the hands are dirty or nonconductive. Also, user with partial physical disability can interact with electronic devices using such systems. Research in this direction has got major boost because of the emergence of low-cost sensors such as Leap Motion, Kinect or RealSense devices. In this paper, we propose a Leap Motion controller-based methodology to facilitate rendering of 2D and 3D shapes on display devices. The proposed method tracks finger movements while users perform natural gestures within the field of view of the sensor. In the next phase, trajectories are analyzed to extract extended Npen++ features in 3D. These features represent finger movements during the gestures and they are fed to unidirectional left-to-right Hidden Markov Model (HMM) for training. A one-to-one mapping between gestures and shapes is proposed. Finally, shapes corresponding to these gestures are rendered over the display using MuPad interface. We have created a dataset of 5400 samples recorded by 10 volunteers. Our dataset contains 18 geometric and 18 non-geometric shapes such as "circle", "rectangle", "flower", "cone", "sphere" etc. The proposed methodology achieves an accuracy of 92.87% when evaluated using 5-fold cross validation method. Our experiments revel that the extended 3D features perform better than existing 3D features in the context of shape representation and classification. The method can be used for developing useful HCI applications for smart display devices.

show abstract

“…Following statistical measures as described in (33)(34)(35)(36) were used for analyzing the results,…”

Section: Analysis Of Resultsmentioning

confidence: 99%

Visual rendering of shapes on 2D display devices guided by hand gestures

2019

View full text Add to dashboard Cite

show abstract

“…According to the review, we found that most of vision based techniques in hand gesture recognition are suitable for either the hand is dynamic or statistic gestures and one or two-handed [10]. Some techniques are not flexible with the conditions.…”

Section: Related Workmentioning

confidence: 99%

Hand Gesture Recognition Algorithm for Smart Cities based on Wireless Sensor

Ganokratanaa

Pumrin

2017

Int. J. Onl. Eng.

View full text Add to dashboard Cite

The relationship between humans and computers is called human computer interaction (HCI). HCI is a major research topic in the area of assistive technology. In the field of medical assistive technology specifically, a hand gesture is considered a suitable method to convey information. It can help elderly people, who are unable to walk or speak, communicate with caregivers whenever they need help. This paper proposes a system to recognize the hand gestures of elderly people using an inexpensive Raspberry Pi. A vision-based algorithm is developed to detect and classify dynamic hand gestures in real time on the Raspberry Pi embedded platform. There are three main procedures; contour detection, convex extraction, and rule-based classification. The system can detect six different gestures on both hands in various orientations. The experiment showed good results in detecting and classifying their meanings as lingual descriptions.

show abstract

“…However a huge database is required to deal with all the possible set of gestures in 3-D based model [1].Elastic graph matching was proposed to detect the gestures from the complex background but this approach was computationally complex and strict with its boundary conditions. The Zernike like moments can be easily used for gesture recognition to get the sample point from the required set of the gesture [2]. The frames preprocessing include skin color segmentation and Zernike like moments to detect the shapes of the gesture.…”

Section: Related Workmentioning

confidence: 99%

“…In order to train the HMM, a Baum-Welch re-estimation algorithm, which adapts the internal states of the HMM according to some feedback concerning the accuracy, was used. Sphinx is tool which is used to make the words from the letters, this tool can be easily refered as the framework to .NET , JAVA and other technologies [2] IV. SYSTEM OVERVIEW Fig.1: System overview As discussed in previous section, the steps are pictured above.…”

Section: Hidden Markov Models the Hidden Markovmentioning

confidence: 99%