Vision-Based Finger Detection, Tracking, and Event Identification Techniques for Multi-Touch Sensing and Display Systems

Abstract

This study presents efficient vision-based finger detection, tracking, and event identification techniques and a low-cost hardware framework for multi-touch sensing and display applications. The proposed approach uses a fast bright-blob segmentation process based on automatic multilevel histogram thresholding to extract the pixels of touch blobs obtained from scattered infrared lights captured by a video camera. The advantage of this automatic multilevel thresholding approach is its robustness and adaptability when dealing with various ambient lighting conditions and spurious infrared noises. To extract the connected components of these touch blobs, a connected-component analysis procedure is applied to the bright pixels acquired by the previous stage. After extracting the touch blobs from each of the captured image frames, a blob tracking and event recognition process analyzes the spatial and temporal information of these touch blobs from consecutive frames to determine the possible touch events and actions performed by users. This process also refines the detection results and corrects for errors and occlusions caused by noise and errors during the blob extraction process. The proposed blob tracking and touch event recognition process includes two phases. First, the phase of blob tracking associates the motion correspondence of blobs in succeeding frames by analyzing their spatial and temporal features. The touch event recognition process can identify meaningful touch events based on the motion information of touch blobs, such as finger moving, rotating, pressing, hovering, and clicking actions. Experimental results demonstrate that the proposed vision-based finger detection, tracking, and event identification system is feasible and effective for multi-touch sensing applications in various operational environments and conditions.

Eye-wearable head-mounted tracking and gaze estimation interactive machine system for human–machine interface

“…This study used grayscale images for preprocessing to increase the amount of iris information obtained. After iris information had been added, an automatic multilevel thresholding technique [14][15][16] obtained the regions of interest for the iris in the image. Mathematical morphology operations of dilation and erosion were then conducted on the thresholded regions.…”

Robust Observation Detection for Single Object Tracking: Deterministic and Probabilistic Patch-Based Approaches

“…Histogram-based tracking algorithms [3,26] have been applied successfully to non-rigid objects because the matching is done based on the statistics of a group of pixels. A multilevel thresholding of the histogram data is used by Chen et al [27] to improve the detection robustness to illumination changes and spurious infrared noise. The most popular histogram-based tracker is a mean shift algorithm [28] where the next location is predicted based on the input of histogram backprojection via the mean shift algorithm.…”

A Touch Sensing Technique Using the Effects of Extremely Low Frequency Fields on the Human Body

“…Many touch sensing technologies have therefore been developed for commercial purposes. Examples include technologies based on infra-red sensing elements [1,2,3,4], resistive [5,6] and capacitive touch panels [7,8,9], cameras [10], the acoustic and deflection characteristics of touch panels [11,12,13], and others [14,15,16]. One of the widely used techniques is the mutual capacitive method, which is used in almost all smartphones and tablets [17].…”