Method for generating real-time interactive virtual fixture for shared teleoperation in unknown environments

Abstract

A virtual fixture (VF) is a constraint built into software that assists a human operator in moving a remote tool along a preferred path via an augmented guidance force to improve teleoperation performance. However, teleoperation generally applies to unknown or dynamic environments, which are challenging for VF use. Most researchers have assumed that VFs are pre-defined or generated automatically; however, these processes are complicated and unreliable in unknown environments where teleoperation is in high demand. Recently, a few researchers have addressed this issue by introducing a user-interactive method of generating VFs in unknown environments. However, these methods are limited to generating a single type of primitive for a single robot tool. Moreover, the accuracy of the VF generated by these methods depends on the accuracy of the human input. Thus, applications of these methods are limited. To overcome those limitations, this work introduces a novel interactive VF generation method that includes a new method of representing VFs as a composition of components. A feature-based user interface allows the human operator to intuitively specify the VF components. The new VF representation accommodates a variety of robot tools and actions. Using the feature-based interface, the process of VF generation is more intuitive and accurate. In this study, the proposed method is evaluated with human subjects in three teleoperation experiments: peg-in-hole, pipe-sawing, and pipe-welding. The experimental results show that the VFs generated by the proposed approach result in a higher manipulation quality while demonstrating the lowest total workload in all experiments. The peg-in-hole task teleoperation was the safest in terms of failure proportion and exerted force of the robot tool. In the pipe-sawing task, the positioning of the robot tool was the most accurate. In the pipe-welding task, the quality of weld was the best in terms of measured tool-trajectory smoothness and visual weld observation.

A Probabilistic Approach to Multi-Modal Adaptive Virtual Fixtures

“…Selvaggio et al [7] detect limit switches and plan VFs for reaching and manipulating these switches. Pruks and Ryu [8] use visual measurements to allow users to interactively define VFs based on geometric primitives, for example cylindrical fixtures for circles detected in the image. Contrarily, Hager [9] and Bettini et al [10] calculate VF forces directly based on camera images for 2D line following thus creating dynamic visual servoing fixtures.…”

“…6b shows that the attractor point is temporarily located between connectors when switching. With our choice of gating function (8), the importance of a connector grows exponentially with decreasing end effector distance, ensuring that the operator is, in the end, always guided towards a connector. Moment matching (10) leads to a large variance of the purple Gaussian in direction of the connectors B .…”

Proxy-based Guidance Virtual Fixtures with Orientation Constraints

“…Virtual fixtures introduce the concept of anisotropy, which refers to different properties exhibited in different directions in materials, into robotics. The implementation of virtual fixtures is a research field with a wide range of applications, including tasks in industrial scenarios such as drilling and polishing [10,11], welding [12], assembly [13], as well as tasks in surgical scenarios such as stitching [14,15] and knotting [16].…”

“…A few works have studied orientation constraints. Orientation constrains are generally limited to a fixed value rather than a range [12,19] . Papageorgiou, D.et al [13] proposed an orientational virtual fixture that establishes different degrees of freedom constraints based on partial knowledge of the task.…”

A humanoid robot teleoperation approach based on waist–arm coordination

“…In terms of dual-arm control, many researchers have conducted research on teleoperation bimanual coordination. Methods based on dynamic movement primitives (Deng et al , 2017; Franzese et al , 2022; Rakita et al , 2019), haptic shared (Selvaggio et al , 2018; Zeestraten et al , 2018), behavior assistance (Pruks and Ryu, 2022; Sun et al , 2020c) and visual recognition (Cisneros et al , 2015; Haynes et al , 2017; Lim et al , 2017) have been used to solve the bimanual coordination problem. However, the original intention of the above methods is to reduce the user’s cognitive burden and operational fatigue.…”