A Virtual Keyboard Based on True-3D Optical Ranging

Abstract

In this paper, a complete system is presented which mimics a QWERTY keyboard on an arbitrary surface. The system consists of a pattern projector and a true-3D range camera for detecting the typing events. We exploit depth information acquired with the 3D range camera and detect the hand region using a pre-computed reference frame. The fingertips are found by analyzing the hands' contour and fitting the depth curve with different feature models. To detect a keystroke, we analyze the feature of the depth curve and map it back to a global coordinate system to find which key was pressed. These steps are fully automated and do not require human intervention. The system can be used in any application requiring zero form factor and minimized or no contact with a medium, as in a large number of cases in human-to-computer interaction, virtual reality, game control, 3D designs, etc.

An integrated bundle adjustment approach to range camera geometric self-calibration

“…where d 0 is the rangefinder offset, which may be coupled to a particular integration time; d 1 is the scale error; d 2 -d 7 are the cyclic or periodic error terms; U is the unit length, which is equal to one-half of the modulating wavelength; and e 1 and e 2 are the signal propagation delay errors (Fuchs and May, 2007), also known as the clock-skew errors (Du et al, 2005). In contrast to Lindner and Kolb (2006), who use B-splines to model the cyclic errors, or Fuchs and May (2007), Schiller et al (2008) and Kim et al (2008) who use algebraic polynomials, the cyclic-error models chosen here are driven by the known physical causes of these periodic effects (e.g.…”

“…May et al (2006) also proposed pixel-wise calibration. For their virtual keyboard finger tracking application, Du et al (2005) employed independent camera and range measurement calibrations. One such procedure used range images of a plane to estimate the clock-skew error correction terms.…”

A ToF-Camera as a 3D Vision Sensor for Autonomous Mobile Robotics

“…There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17,18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20], 3D sensing for automated vehicle guidance and safety systems, wheelchair assistance [21], outdoor surveillance [22], simultaneous localization and mapping (SLAM) [23], map building [24], medical respiratory motion detection [25], robot navigation [26], semantic scene analysis [27], mixed/augmented reality [28], gesture recognition [29], markerless human motion tracking [30], human body tracking and activity recognition [31], 3D reconstruction [32], domestic cleaning tasks [33] and human-machine interaction [34,35].…”

A virtual keyboard system based on Multi-Level Feature Matching

“…On the other hand, the interdigital clefts are shown as the valleys of the contour and can be detected by finding the local minima of the k-curvature. K-curvature computation can be simplified by using the approximation of sign change [18], as shown in Fig 5. After the fingertips are detected in 2D image plane, they are mapped back into the 3D space using the depth map. …”