Haptic interaction with 2D images

Abstract

Visual and haptic rendering pipelines exist concurrently and compete for computing resources while the refresh rate of haptic rendering is two orders of magnitude higher than that of visual rendering (1000 Hz vs. 30-50Hz). However, in many cases, 3D visual rendering can be replaced by merely displaying 2D images, thus releasing the resources to image-driven haptic rendering algorithms. These algorithms provide for haptic texture rendering in vicinity of a touch point, but usually require additional information augmented with the image to provide for haptic perception of geometry of the shapes displayed in images. We propose a framework for making tangible images which allows haptic perception of three features: scene geometry, texture and physical properties. Haptic geometry rendering technique uses depth information, that could be acquired by a multitude of ways for providing haptic interaction with images and videos in real-time. The presented method neither performs 3D reconstruction nor requires for using polygonal models. It is based on direct force calculation and allows for smooth haptic interaction even at object boundaries. We also propose dynamic mapping of haptic workspace in real-time to enable sensation of fine surface details. Alternately, one of the existing shading-based haptic texture rendering methods can be combined with the proposed haptic geometry rendering algorithm to provide believable interaction. Haptic perception of physical properties is achieved by automatic segmentation of an image into haptic regions and interactive assignment of physical properties to them

Forbidden region virtual fixtures from streaming point clouds

“…Haptic interaction with streaming point clouds has been investigated by several groups [27][28][29]. The approach has typically been to iteratively move the proxy toward the HIP while satisfying the non-penetration constraint (formed by the point cloud).…”

Image-Driven Haptic Rendering in Virtual Environments

“…These depth values are used to calculate the visible surface normal at the pixel visited by the HIP. Since the force is applied in the direction of the surface normal, it is normalized and scaled by the penetration depth to get the feedback force [31][32]. Several adjustments are made to the calculated forces to remove inconsistencies in the estimated depth map due to noise and provide stability.…”

Image-inspired haptic interaction

“…The 2*2 contingency tables are formed by classifying the number of the users in the experimental group based on their ratings for each question and their genders. Since all the questions were rated on a scale of 1 to7, the column category is determined based on whether the rating falls in the range of [1,4) or [4,7]. The Fisher's exact test results for each table are also presented in Appendix B.…”

“…Digital images are usually made tangible in the following three ways. Firstly, the haptic forces can be derived directly from the image by analyzing pixel information [1,2].…”