Augmented reality for enhanced visual inspection through knowledge-based deep learning

Abstract

A two-stage knowledge-based deep learning algorithm is presented for enabling automated damage detection in real-time using the augmented reality smart glasses. The first stage of the algorithm entails the identification of damage prone zones within the region of interest. This requires domain knowledge about the damage as well as the structure being inspected. In the second stage, automated damage detection is performed independently within each of the identified zones starting with the one that is the most damage prone. For real-time visual inspection enhancement using the augmented reality smart glasses, this two-stage approach not only ensures computational feasibility and efficiency but also significantly improves the probability of detection when dealing with structures with complex geometric features. A pilot study is conducted using hands-free Epson BT-300 smart glasses during which two distinct tasks are performed: First, using a single deep learning model deployed on the augmented reality smart glasses, automatic detection and classification of corrosion/fatigue, which is the most common cause of failure in high-strength materials, is performed. Then, in order to highlight the efficacy of the proposed two-stage approach, the more challenging task of defect detection in a multi-joint bolted region is addressed. The pilot study is conducted without any artificial control of external conditions like acquisition angles, lighting, and so on. While automating the visual inspection process is not a new concept for large-scale structures, in most cases, assessment of the collected data is performed offline. The algorithms/techniques used therein cannot be implemented directly on computationally limited devices such as the hands-free augmented reality glasses which could then be used by inspectors in the field for real-time assistance. The proposed approach serves to overcome this bottleneck.

Technology Behavior Model—Beyond Your Sight with Extended Reality in Surgery

“…This leads to subjective changes in user behavior changes. The flexibility of the model changes to facilitate the comprehensiveness of the research, as relevant theoretical structures can be added according to the actual needs of the new technology under study [52]. Based on this model, the "Extended Reality Technology Behavior Model" is designed by combining the extended reality smart glasses.…”

An Outlook on the Future Marine Traffic Management System for Autonomous Ships

“…In other cases, the problem is the opposite. There are an enormous amount of data, and the available machines are limited, which is why [38] they propose a deeplearning algorithm that works in two phases, thus reducing the computational load of the single-stage. The development of the trend towards a synergistic use leads to the need to have a unified interface between the various technologies used.…”

Opportunities and Challenges of Smartglass-Assisted Interactive Telementoring

“…Moreover, establishing sufficient network bandwidth for high-resolution videos is also important to share the view and contents [51] of mentors or mentees without interruptions and delays. As more interaction data between mentors and mentees are accumulated, we can benefit from deep learning approaches [52][53][54] that enhance various steps involved in the XR pipeline (i.e., registration, tracking, rendering and interaction, task guidance [55,56]).…”