Alfred: the Haptic Butler On-Demand Tangibles for Object Manipulation in Virtual Reality using an ETHD

Abstract

We present "Alfred", a novel haptic paradigm for object manipulation in 3D immersive virtual reality (VR). It uses a robotic manipulator to move tangible objects in its workspace such that they match the pose of virtual objects to be interacted with. Users can then naturally touch, grasp and manipulate a virtual object while feeling congruent and realistic haptic feedback from the tangible proxy. The tangible proxies can detach from the robot, allowing natural and unconstrained manipulation in the 3D virtual environment (VE). When a manipulated virtual object comes into contact with the virtual environment, the robotic manipulator acts as an encounter-type haptic display (ETHD), positioning itself so as to render reaction forces of the environment onto the manipulated physical object. Here, we discuss the concept for this novel approach and present a simplified prototype using a single detachable tangible proxy supported by a UR5 industrial robot. Through illustrative use-cases in VR and a preliminary performance evaluation, we discuss implications for robot control and design of interaction techniques. We show that Alfred is adaptable to a wide range of virtual environments and interaction scenarios, making it a promising approach for haptic-enabled manipulation in VR, although system latency is a limitation that still remains to be addressed.

"Let’s Meet and Work it Out": Understanding and Mitigating Encountered-Type of Haptic Devices Failure Modes in VR

“…It is felt that a large number of useful applications, in manufacturing design, and design verification, are amenable to this approach with existing technology." [37] McNeely hence evokes using <virtual:physical> mappings (Figure 9): as a thorough 1:1 mapping is more complex to build, a physical object can for instance overlay multiple virtual ones [22,40]. Advances in Pseudo-Haptics 4 demonstrated that objects of approximately similar primitives can be used to simulate the same object, without altering the user's perception.…”

RoboHapalytics: A Robot Assisted Haptic Controller for Immersive Analytics

“…For example, Lee et al [43] developed a remote control approach to manipulate the robot end effector movement with hand gestures through virtual reality controllers. Robot arms are also widely used in encountered-type haptics to continuously follow and match the user's actions [3,22,52,72]. For example, Araujo et al [3] used a robot arm to provide feeling of textures through a tactile device on a robot arm and simple interactions through buttons.…”

“…For example, Araujo et al [3] used a robot arm to provide feeling of textures through a tactile device on a robot arm and simple interactions through buttons. Mercado et al [52] used a robot arm to provide an on-demand tangible surface by placing a physical plane under the contact area when the user is putting an object on a surface in the virtual world. Such approaches have been evaluated and the results seem to indicate that they enhance immersive experiences as they provide real-time "being there" sensation when interacting with virtual objects [3,29,52].…”

“…While other devices such as drones [1,76] or other custom robots [29,67] could be used for haptics, they tend to restrict the direction or angles that the device can eventually reach. Robot arms, however, are more versatile with a high load-bearing ability and have thus even been used to simulate large objects such as walls or tables in virtual experiences [52]. The speed at which they operate allows for the representation of several objects (see, e. g., [52]).…”

“…Robot arms, however, are more versatile with a high load-bearing ability and have thus even been used to simulate large objects such as walls or tables in virtual experiences [52]. The speed at which they operate allows for the representation of several objects (see, e. g., [52]). In addition, the detachable head design of commercial robot arms increases the variety of haptic feedback available for a single encountered haptic feedback system [3].…”