Hand Gesture based Concept of Human - Mobile Robot Interaction with Leap Motion Sensor

Ahmed, Sevil; Popov, Vasil L.; Topalov, Andon V.; Shakev, Nikola

doi:10.1016/j.ifacol.2019.12.543

Cited by 13 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A nose-teeth based mobile interaction system employing Viola-Jones and Lucas Kanade optical flow algorithm has been proposed and implemented in this research to control almost all the gesture and global operations of a smartphone using nose and teeth. Most of the existing systems focus on the interactions with desktop computer using different gestures [1], [2], [4], [5], [7], [8], [13], [21], BCI [28], [30], or person's breath [29]. There are a very few studies focusing on smartphone interaction [6], [23].…”

Section: Discussionmentioning

confidence: 99%

“…Therefore, considering all this, we developed the nose-teeth mobile interaction system which will stretch a helping hand to the people who are incapable of using the android smartphones due to their disabilities. Some other hardware based systems are proposed for interacting with other devices like robot, 3DTV in VR environment and these are based on additional hardware devices and sensors [30]- [33], [42], which are costly and maybe difficult to afford for some people. Any new efficient solution based on an expensive device will not be able to effectively help the disabled people.…”

Section: Discussionmentioning

confidence: 99%

“…A few other recently published articles focused on different ways to interact with other devices such as controlling robots remotely, interacting with a virtual environment, controlling 3DTV, etc. For example, in [30], a system is proposed to control a robot from a distance using hand gestures in LMC (Leap Motion Controller) on the desktops. Similarly, in [31], BCI and AR technologies are combined together to interact with robots remotely.…”

Section: Interaction With Other Devicesmentioning

confidence: 99%

See 2 more Smart Citations

Developing a Novel Hands-Free Interaction Technique Based on Nose and Teeth Movements for Using Mobile Devices

Islam¹,

Aadeeb²,

Khan³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Human-mobile interaction is aimed at facilitating interaction with the smartphone devices. The conventional way to interact with mobile devices is through manual input where most of the applications are made assuming that the end user has full control over their hand movements. However, this assumption excludes people who are unable to use their hands or have suffered limb damage. In this paper, we proposed a nose and teeth based interaction system, which allows the users to control their mobile devices completely hands free. The proposed system uses the front facing camera of the smartphone to track the position of the nose for cursor control on the smartphone screen. The system detects teeth for performing the touch screen events such as tap, scroll, long press, and drag. Viola-Jones algorithm is used to detect the face and teeth based on the Haar features. After detecting the face, the nose position is calculated and tracked continuously using Lucas Kanade's method for optical flow estimation. All the touch screen events have been implemented in the system so that the user can execute all the operations of the smartphone. To evaluate the performance and the effect of (smartphone) device type on the execution time, the proposed system was installed in 3 smartphone devices and 7 trials for each device were performed by 3 different able-bodied elderly persons. The result shows a significant success rate for the detection of nose and teeth, and for the execution of the operations. The execution time of each operation slightly varies by 0.72s on average because of the configuration of the smartphones.INDEX TERMS HCI, human-mobile interaction, gesture operations, disabled user, accessibility, mobile device, smartphone.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Interaction With Other Devicesmentioning

confidence: 99%

See 1 more Smart Citation

Developing a Novel Hands-Free Interaction Technique Based on Nose and Teeth Movements for Using Mobile Devices

Islam¹,

Aadeeb²,

Khan³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The recognition systems of hand gesture based on centroid points [26] are developed to control a mouse [27], and to operate a video player [28]. The hand gesture from leap motion is also used to control mobile robot [29], robot manipulator [30], flight simulation [21] and detect hand tremor [19]. Besides for head gestures [3], orientation features are also used to recognize finger signs [24].…”

Section: Introductionmentioning

confidence: 99%

“…Subsequently, wrist orientation is found to be an alternative feature in recognizing hand gestures [23]. Wrist rotation, especially pitch and yaw variables, can be used to control mobile robot [29]. It is shown that wrist orientation can be an alternative feature to recognize hand gestures [32], which can be used to lessen on finger dependencies.…”

Section: Introductionmentioning

confidence: 99%

Electric Wheelchair Control Using Wrist Rotation Based on Analysis of Muscle Fatigue

et al. 2022

View full text Add to dashboard Cite

The wheelchair is one of the most common assistive technologies for people with motor impairment, due to its environmentally friendly features in terms of mobility and comfort. However, the operational method for conventional wheelchair is still inconvenience for people with finger problems. Therefore, in the project , a novel control method to operate an electric wheelchair, using hand gestures ( wrist rotation) is developed. Sixty-five (65) participants were involved in this study. Five hand gestures were considered and studied for forward, backward, right, left, and stop maneuvers while considering human ergonomics factor. In this study, the stop maneuver was determined based on the most comfortable hand position to mitigate a fatigue experience, with two gesture classifying methods further investigated. The first method based on threshold has a promising accuracy of 96%, and 91% precision. This method however requires a calibration every time a new user is introduced. The second method, a Naïve Bayes approach, was observed to solve the problem, as it has about 99% of both the accuracy and the precision. The evaluation of this method was then conducted with six participants that operated the wheelchair to follow the trajectories from start to end. The results showed that the participants comfortably controlled the developed wheelchair system to the goal without any collision. Results from experiments indicate that the proposed approach has high accuracy and the potential to solve the problem related to finger dependencies and hand fatigue.

show abstract

Maneuvers Under Estimation of Human Postures for Autonomous Navigation of Robot KUKA YouBot

Gordón

Barahona

Cumbajín

et al. 2020

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Hand Gesture based Concept of Human - Mobile Robot Interaction with Leap Motion Sensor

Cited by 13 publications

References 5 publications

Developing a Novel Hands-Free Interaction Technique Based on Nose and Teeth Movements for Using Mobile Devices

Developing a Novel Hands-Free Interaction Technique Based on Nose and Teeth Movements for Using Mobile Devices

Electric Wheelchair Control Using Wrist Rotation Based on Analysis of Muscle Fatigue

Maneuvers Under Estimation of Human Postures for Autonomous Navigation of Robot KUKA YouBot

Contact Info

Product

Resources

About