A Vision-Based Remote Control

Abstract

This chapter presents a vision-based system for touch-free interaction with a display at a distance. A single camera is fixed on top of the screen and is pointing towards the user. An attention mechanism allows the user to start the interaction and control a screen pointer by moving their hand in a fist pose directed at the camera. On-screen items can be chosen by a selection mechanism. Current sample applications include browsing video collections as well as viewing a gallery of 3D objects, which the user can rotate with their hand motion. We have included an up-to-date review of hand tracking methods, and comment on the merits and shortcomings of previous approaches. The proposed tracker uses multiple cues, appearance, color, and motion, for robustness. As the space of possible observation models is generally too large for exhaustive online search, we select models that are suitable for the particular tracking task at hand. During a training stage, various off-the-shelf trackers are evaluated. From this data different methods of fusing them online are investigated, including parallel and cascaded tracker evaluation. For the case of fist tracking, combining a small number of observers in a cascade results in an efficient algorithm that is used in our gesture interface. The system has been on public display at conferences where over a hundred users have engaged with it.

User-centred process for the definition of free-hand gestures applied to controlling music playback

“…Nevertheless, similar gestures for next track and previous track are used in our approach and Moove. Stenger et al [28] present a remote control based on a single camera mounted on a public display, which is controlled by hand gestures. The system is aimed for public settings in particular and can be used to control several applications, including a system to browse video collections as well as viewing a gallery of 3D objects.…”

Free-hand gestures for music playback

“…In contrast to the approach presented in this paper, the system uses a set of four predefined hand gestures: left-to-right and right-to-left motions, covering the camera and tapping the camera. In [21] Stenger et al present a remote control based on a single camera mounted on a public display, which is controlled by hand gestures. The system is aimed for public settings in particular and can be used to control several applications, including a system to browse video collections as well as viewing a gallery of 3D objects.…”