Effectiveness of Augmented Reality Guides for Blind Insertion Tasks

Abstract

Although many augmented reality (AR)-based assembly support systems have been proposed in academic research and industry, the effectiveness of AR to resolve the occlusion issue in the context of a blind assembly process remains an unexplored topic. Therefore, the present work investigates how AR can assist operators during the execution of blind manual assembly tasks. Specifically, an AR research set-up was designed to provide assistance in occlusion situations during a peg-in-hole task. The set-up featured a see-through device (HoloLens), which provides operators with two modes of visual augmentations that directly overlay on the assembly objects. The first mode referred to as the “wireframe overlay” displays the inner part of the objects, providing an inside view of the occluded parts, and the second one referred to as the “axes overlay,” displays the axes of the objects and their slots, indicating how to align the different parts during the assembly. The effectiveness of these AR visualizations was compared to a baseline augmentation-free situation in a controlled experiment. Thus, following a within-subject design, 30 participants performed a two-stages blind insertion task. Their performances represented by task completion time, insertion errors, and smoothness of the insertions were recorded. In addition, a post-questionnaire reported their subjective perception of task difficulty during the task and their preferences. Results indicated a strong acceptance of participants for AR visualizations that they rated as allowing them to perform the task more easily. However, no statistically significant differences in terms of objective performance measures were found. Yet, it was found that axes overlay produced smoother trajectories compared to the wireframe overlay, highlighting the potential effect of more abstract visualization aids.

AR-Assisted Guidance for Assembly and Maintenance of Avionics Equipment

“…In terms of AR assembly guidance in the blind area of the visual field, Khenak [34] used AR to provide assembly guidance of an occluded part in a blind insertion task for an assembled object. Feng [35] proposed an AR assembly guidance method to provide auxiliary guidance in a free-hand bolt assembly task in a blind area.…”

A blind area information perception and AR assembly guidance method based on RGBD data for dynamic environments and user study

“…Henderson et al [18] developed an AR-based assisted assembly system to address the issues of confined space and mutual obscuration of parts in the military vehicle maintenance process, which presents assembly information in the form of AR text, arrows, or animations, effectively reducing the operator's head rotation. Khenak et al [19] developed experiments to compare two approaches for data visualization for the blind insertion task. The first is a wireframe overlay mode, which shows the model wireframe, and the second is an axis overlay mode, which only shows essential geometric constraint elements (e.g., axes).…”

Multimodal markers for technology-independent integration of augmented reality devices and surgical navigation systems

“…All of these issues lead to cognitive overload which makes the use of navigation systems difficult for surgeons, especially those in training. Augmented reality (AR) has been used previously to support in various applications, such as assisting with assembly tasks (Westerfield et al 2015;Khenak et al 2020), to address the aforementioned limitations. It allows the integration of virtual objects, such as surgical plannings or critical structures, in the field of view of the user.…”