Kick-up menus

Abstract

Interaction with mobile applications is often awkward due to the limited and miniaturized input modalities available. Kick-Up Menus exploit the video capabilities of camera equipped smart-phones and PDA's to provide a fun solution for interaction tasks like menu selection and simple games. THE CONCEPTKick-Up Menus can be implemented on video capable mobile devices. The camera facing towards the back of the mobile device ( Fig.1, right) is used to detect "kicking" movements of the user's feet. When a collision between the user's "kick" and an interaction object shown on the screen of the mobile device is detected, a corresponding interaction event for the application is generated. Figure 1.: Kick-Up-MenusAs shown in Fig.1 interaction objects can be simple menu labels. In these "kick-up-menus" users can open/close submenus or trigger actions by kicking at the corresponding labels. In this case only the horizontal coordinate of the "kick" is used to determine the desired label. The technique is not is not limited to menu selection. The direction and speed of the detected motion can also be exploited.To demonstrate this we have developed a small game called AR-Soccer (Fig.2 ). In the game the player has to kick a virtual ball (a moving interaction object) with his real foot into a virtual goal, bypassing the goalkeeper. Figure 2: AR-Soccer and the edge-tracking algorithm IMPLEMENTATIONOur computer vision component calculates the motion of a "kicking object" (e.g. the user's foot) from the incoming video-stream and detects collisions between it and the interaction objects. Since current mobile devices lack highperformance processors we have developed a simple and fast algorithm that combines 2D edge extraction and tracking and operates only in regions of interest (ROI) around the interaction objects [1]. To perform the collision detection, straight edges inside the ROI are vectorized and tracked between two consecutive images (Fig. 2, right). Then the median direction and speed of edge motions inside the ROI is computed. If it points towards an interaction object an interaction event is transmitted to the application OUTLOOK While current hardware limits our system to the processing of 5-7 frames/second, suggesting its use as part of games where the interaction is part of the challenge, feedback from users of the AR-Soccer application has been very positive. We hope that the creation of faster and more robust CV based interaction techniques for productivity applications will become possible on the next generation of PDAs.

An Investigation on Selection Mechanisms for Mobile Camera-based Cursor Manipulation

“…Hachet and colleagues [11] presented an interface which tracked the movement of a rectangle target to interact with 2D or 3D virtual contents on a mobile device's display. Parlke et al [12] made use of kicking actions detected through camera input to conduct mobile device menu manipulation. Gallo et al [13] and Baldauf et al [14] respectively presented mobile camera-based interfaces using fingertip detection techniques.…”