Multi-Phase Multi-Modal Haptic Teleoperation

Abstract

Virtual Fixtures facilitate teleoperation, for instance by guiding the human operator. Developing these Virtual Fixtures in tasks with tight tolerances remains challenging. Fixtures with a high stiffness allow for more precise guidance, whereas a lower stiffness is required to allow for corrections. We observed that many assembly operations can be split into different phases -approaching, positioning, in-contact manipulation -each with different accuracy requirements. Therefore, we propose to use multi-modal fixtures, satisfying the different requirements of these phases: i.e. a position-based Trajectory Fixture for approaching and a more accurate Visual Servoing Fixture for the positioning phase. A state estimation and arbitration component ensures smooth transitions between the fixtures to provide optimal support for the operator and to achieve global availability paired with local precision at the same time. It also allows a high stiffness to be used throughout, thus achieving good guidance for all phases. The approach is validated in an application from a space scenario, consisting of the assembly of a CubeSat subsystem. The empirical results from a pilot study on this task show that our approach is faster and requires less interaction force from the operator than the baseline method.

A Probabilistic Approach to Multi-Modal Adaptive Virtual Fixtures

“…We evaluate our method on the use case of CubeSat subsystem assembly [3]. We empirically set l x = l y = l z = 0.06, l wx = l wy = l wz = 0.2, γ = 1 × 10 −20 , κ = 3 × 10 3 and η = 1×10 −6 for stiffness scaling in the visual servoing fixture.…”

“…Manually tuned scaling factors allow them to have all fixtures active at the same time. In our previous work [3] we hand-designed an arbitration function between position-and vision-based fixtures. A limitation of such approaches is that smooth switching between target poses requires the handcrafting of either a stabilizing controller or an arbitration function.…”

“…Using an in-hand camera leads to an increasing accuracy when approaching the target as the connector's size in pixels increases. Previously [3], we used a fixed grayscale threshold to binarize the intensity image I and extract targets using OpenCV [32] rectangle extraction. Depending on illumination conditions and camera settings, the optimal threshold differs.…”

“…Note that this corresponds to the second moment of the multi-modal distribution [13], unlike [21] where by omitting the vector outer product Log µVS (µ m ) Log µVS (µ m ) it was the result of a linear combination of Gaussians. Under (9), we use µ VS as the attractor point in (3). Due to our choice of h m , (10) matches Σ m in the vicinity of connector m, increasing as the end effector moves away.…”

“…An insertion task typically requires an approach phase, where trajectory guidance is needed, followed by an insertion phase, where visual guidance takes over. Position and vision [3] or even forces [4] are examples of fixture input modalities. The arbitration of different modalities, i.e.…”