Camera-Based Motion Recognition for Mobile Interaction

Abstract

Multiple built-in cameras and the small size of mobile phones are underexploited assets for creating novel applications that are ideal for pocket size devices, but may not make much sense with laptops. In this paper we present two vision-based methods for the control of mobile user interfaces based on motion tracking and recognition. In the first case the motion is extracted by estimating the movement of the device held in the user's hand. In the second it is produced from tracking the motion of the user's finger in front of the device. In both alternatives sequences of motion are classified using Hidden Markov Models. The results of the classification are filtered using a likelihood ratio and the velocity entropy to reject possibly incorrect sequences. Our hypothesis here is that incorrect measurements are characterised by a higher entropy value for their velocity histogram denoting more random movements by the user. We also show that using the same filtering criteria we can control unsupervised Maximum A Posteriori adaptation. Experiments conducted on a recognition task involving simple control gestures for mobile phones clearly demonstrate the potential usage of our approaches and may provide for ingredients for new user interface designs.

Camera-Based Gesture Tracking for 3d Interaction Behind Mobile Devices

“…28,31 Due to the hardware capabilities of new smartphones and tablet PCs, results of the recent research works, have been implemented on mobile devices. Accelerometerbased approaches recognize hand gesture motions by using the device's acceleration sensor, 1,2,7,8,16 use visual color markers for detecting the¯ngertips to facilitate the gesture-based interaction in augmented reality applications on mobile phones, 3,24 perform marker-less visual¯ngertip detection, based on the color analysis and computer vision techniques for manipulating the applications in human mobile device interaction, 6 use color-based face tracking to interact with mobile applications, 14 perform HMM to recognize di®erent dynamic hand gesture motions, 12,13 use visual marker or shape recognition to augment and track the virtual objects and graphical models in augmented reality environments. Since the computer vision techniques are extensively developed for advanced pattern recognition problems, we do not need to reinvent the wheels for our purpose.…”