Application of feature tracking in a vision based human machine interface for Xbox

Adaptive Kalman Filter Applied to Vision Based Head Gesture Tracking for Playing Video Games

“…They need an intuitive and hands-free human machine interface (HMI) to work with computer systems. This problem is escalated for pupils with cerebral palsy (CP), encephalitis or upper limb disabilities in using the controller of video games such as the Xbox [1]. To play games, they typically need a hands-free interface that does not require direct physical contact and motor motions.…”

“…It could be used as an alternative to a physical gamepad and provides hands-free HMI to interact with video games without massive upper limb motions. This helps people with motor disability to interact with the game using head gestures [1][2][3].…”

“…It detects the flow of hundreds of feature points in consecutive images, which, if properly adjusted, can be used to estimate motion information. For instance, it could be used to recognize head gestures in consecutive frames supplied by a camera [1]. Optical flow naturally involves huge data processing and this is a challenge for real-time applications.…”

“…This makes the head return to its original pose immediately after any gesture. This requires fast and precise recognition, in which any tiny head movements are spotted before interruption [1]. The method proposed by Jia et al [13] could not spot small head movements, since it works based on nose position in the center of the face, and this needs to be large enough.…”

“…The experiment is designed to evaluate the improvement made by AKF applied to the vision-based interface while playing a video game. This paper is an extension of our previous work [1], where we developed head gesture-based HMI as an alternative to gamepads for the Xbox game.…”