Integrating Real-Time Object Detection into an AR-Driven Task Assistance Prototype: An Approach Towards Reducing Specific Motions in Therbligs Theory

Abstract

Due to challenges in filling vacant positions and the heightened demands posed on existing staff, employers and project managers are progressively considering the recruitment of inexperienced individuals and seeking strategies to swiftly provide them with essential job-specific knowledge. The potential of industrial AR has been widely researched to support workers in overcoming skill-related knowledge and enhancing industrial processes. However, most studies focus on demonstrating technology usability across different processes and overcoming engineering hurdles on a case-by-case basis. There is no direct benefit analysis on how AR assists construction tasks at human motion level, and how to eliminate the ineffective motions and reduce the duration of effective motions. To fill this gap, this paper first establishes an AR-based near real-time object detection system of small tools and components involved in task processes for egocentric perception of workers in the construction industry. Later, the Standard Operating Procedure (SOP) for scaffolding assembly activities is deconstructed from a manual process into Therbligs-based elemental motions. Finally, this research conducted a comparative study of two prototypes across four dimensions of evaluation. As a step forward in this direction, this paper renews the connotations of Therbligs theory under industry 5.0 era, rethinks the AR-assisted construction task processes, and applies appropriate technologies enhancing the adaptability of AR technology for construction workers’ needs