A Multi-touch Surface Using Multiple Cameras

Abstract

Abstract. In this paper we present a framework for a multi-touch surface using multiple cameras. With an overhead camera and a side-mounted camera we determine the fingertip positions. After determining the fundamental matrix that relates the two cameras, we calculate the three dimensional coordinates of the fingertips. The intersection of the epipolar lines from the overhead camera with the fingertips detected in the side camera image provides the fingertip height. Touches are detected when the measured fingertip height from the touch surface is zero. We interpret touch events as hand gestures which can be generalized into commands for manipulating applications. We offer an example application of a multi-touch finger painting program.

A single camera based floating virtual keyboard with improved touch detection

“…It is accurate and robust; however it requires a 3D optical imaging system. Similarly in [2], [3], the systems provide special features such as hand gestures and multi -touch but [2] requires multiple cameras, and [3] requires unique hardware. In [4], the shadow of a finger is detected, and when it is occluded by the finger, a touch is assumed.…”

3D fingertip and palm tracking in depth image sequences

“…They can be used in combination with inverse kinematics solver for hand pose estimation [1]. Their trajectories can be used for gesture recognition [6,4] or manipulative purpose in multitouch systems [5]. A lot of work have been done for visionbased fingertip tracking, while many previous methods only focus on extracting 2D fingertips and cannot track fingertips robustly for a freely moving hand [6,4,5,9].…”

“…Most previous fingertip tracking schemes are based on contour analysis of the extracted hand region [6,4,5,2] and usually can track the fingertips for only stretched fingers. In [6] fingertips are tracked for infrared image sequences.…”

“…Their trajectories can be used for gesture recognition [6,4] or manipulative purpose in multitouch systems [5]. A lot of work have been done for visionbased fingertip tracking, while many previous methods only focus on extracting 2D fingertips and cannot track fingertips robustly for a freely moving hand [6,4,5,9]. Research Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”

Infrared Sensitive Camera Based Finger-Friendly Interactive Display System

“…A vision based noncontact interactive advertisement with a display wall [5] enables the user to interact with advertisement on a large display wall without any physical contact. Itai Katz et al [6] developed a multi-touch surface using multiple cameras. This system determines the fingertip position with overhead camera and a side-mounted camera and calculates the 3-D coordinates of the fingertips.…”