Human-as-a-Robot Performance in Augmented Reality Teleultrasound

Abstract

<p> <strong>Purpose:</strong> In “human teleoperation” [1], augmented reality (AR) and haptics are used to tightly couple an expert leader to a human follower. To determine the feasibility of human teleoperation, we quantify the abil- ity of humans to track a position and/or force trajectory via AR cues. The human response time, precision, frequency response, and step response were characterized, and several rendering methods were compared. </p> <p><strong>Methods:</strong> Volunteers (n=11) performed a series of tasks as the fol- lower in our human teleoperation system. The tasks involved tracking pre-recorded series of motions and forces, each time with a different rendering method. The order of tasks and rendering methods was ran- domized to avoid learning effects and bias. The volunteers then performed a series of frequency response tests and filled out a questionnaire. </p> <p><strong>Results:</strong> Rendering the full ultrasound probe as a position target with an error bar displaying force led to the best position and force tracking. Following force and pose simultaneously was more difficult but did not lead to significant performance degradation versus following one at a time. On average, subjects tracked positions, orientations, and forces with rms tracking errors of 6.2 ± 1.9 mm, 5.9 ± 1.9˚, 1.0 ± 0.3 N, steady-state errors of 2.8 ± 2.1 mm, 0.26 ± 0.2 N, and lags of 345.5 ± 87.6 ms respectively. Performance decreased with input frequency, until the person could no longer follow, depending on the input amplitude. </p> <p><strong>Conclusion:</strong> This paper characterizes human tracking ability in aug- mented reality human teleoperation, which shows the system’s feasi- bility and good performance, and is important for designing future human computer interfaces using augmented and virtual reality. </p>

Low-Profile 6-Axis Differential Magnetic Force/Torque Sensing

“…The -3 dB cut-off frequency was set to 400 Hz since interactions with stiff environments can lead to high force bandwidths. The phase delay is < 1 • until approximately 10 Hz, which is the maximum typical for forces applied by the human hand [46].…”

“…The master board takes in all the readings from the SMs, combines and processes them on an ESP32-WROOM-32e microcontroller, and sends the resultant force/torque measurements to the host PC or other device. An inertial measurement unit (IMU) is also included on the board for pose estimation [44], and communicates over I2C with the ESP32. The microcontroller has two cores, allowing parallel processing, and two SPI drivers.…”

Robust Object Pose Tracking for Augmented Reality Guidance and Teleoperation

“…From tests it was found that typical hand motion during tracking was slow enough that the sampling rate of the vision system was sufficient. Indeed, the human hand bandwidth for applying forces is approximately 7 Hz [59], and the vision system runs at 22 Hz (see below). The vision system is thus beyond the Nyquist rate for hand motions.…”