DOLPHIN: A Framework for the Design and Perceptual Evaluation of Ultrasound Mid-Air Haptic Stimuli

Abstract

Ultrasound mid-air haptic interfaces can display highly reconfigurable vibrotactile shapes in mid-air for human-computer interaction (HCI) applications. The choice of stimulus shape, spatial, temporal and modulation parameters yields a complex design space, yet relatively little is known about the impact of these design choices on perceived stimulus properties. We define the combination of a spatial discretization of an abstract shape and a set of rules for the temporal display order and intensity modulation of the resulting points as a sampling strategy. We developed DOLPHIN, an open-source framework to aid in designing mid-air stimuli. DOL-PHIN allows the study of the impact of rendering parameters on perceived stimulus properties. This platform-agnostic framework standardizes stimulus descriptions as a step toward more replicability and easier communication in the field. It enables reproduction of stimuli between perceptual experiments and ensures stimuli used in applications correspond to those evaluated in prior perceptual studies. We validated DOLPHIN's usability by conducting a user study assessing the impact of sampling strategy design on curvature discrimination for dynamic mid-air haptic stimuli. The Weber fractions for just-noticeable differences (JNDs) in curvature were found to range between 1 and 1.4, yet no significant effect of the number of spatial sampling points on curvature discrimination was found. This result shows a practical use-case for DOLPHIN and provides insight into rendering mid-air haptic curvature.

Ultrasound Mid-Air Tactile Feedback for Immersive Virtual Reality Interaction

“…the stimuli used in the perception experiments discussed in Sec. 4.2 [31,61]), to use pre-defined complex stimulus patterns that are triggered by events in the VR interaction (e.g. button-press feedback in [37] or the sensations corresponding to supernatural phenomena in [51]), or to extract frame-rate-independent characteristic values from the tracking input from which UMH feedback can be procedurally generated at whichever frequency is best suited to the UMH device.…”

“…To determine the usable ranges and discernible levels of UMH stimulus parameters, psychophysics studies are conducted with the aim of determining detection and discrimination thresholds. Several studies have focused on this, with some of our own work on the topic [31,50,61] discussed in more detail in the present section, however given the complexity of UMH stimuli there is still much to be discovered on this subject. We provide an in-depth overview of the state of the art on the subject in the work published by Mulot et al [61].…”

“…Several studies have focused on this, with some of our own work on the topic [31,50,61] discussed in more detail in the present section, however given the complexity of UMH stimuli there is still much to be discovered on this subject. We provide an in-depth overview of the state of the art on the subject in the work published by Mulot et al [61].…”

“…Ultrasound devices are capable of delivering a variety of sensations ranging from light pressure and vibrotactile stimuli [8] to sensations of airflow [26] and thermal stimuli [39], all within a continuous 3D workspace and with a very high temporal resolution. These high-dimensional stimuli are potentially capable of encoding a lot of information through rapidly and finely re-configurable spatial, temporal and intensity characteristics [61]. The device's workspace and lack of physical constraints on the stimuli means that stimuli do not necessarily need to be restricted to a single point on the skin or even just the hand, but can potentially target any exposed skin on the user's body [24,79].…”

Mid-Air Haptics: Future Challenges and Opportunities

“…An important forward step to that end is the availability of easy-to-use software tools that can facilitate psychophysical studies, enabling researchers who are not experts in the technology to design and output consistent mid-air haptic stimuli. While there exist some attempts in this direction [19], experimental platforms and frameworks for running perceptual studies on ultrasound mid-air haptics are still rare. This section has so far reflected on the challenges of using perceptual knowledge to inform the design and presentation of haptic sensations.…”