Empirical assessment of a RGB-D sensor on motion capture and action recognition for construction worker monitoring

Abstract

Background: For construction management, data collection is a critical process for gathering and measuring information for the evaluation and control of ongoing project performances. Taking into account that construction involves a significant amount of manual work, worker monitoring can play a key role in analyzing operations and improving productivity and safety. However, time-consuming tasks involved in field observation have brought up the issue of implementing worker observation in daily management practice. Methods: In an effort to address the issue, this paper investigates the performances of a cost-effective and portable RGB-D sensor, based on recent research efforts extended from our previous study. The performance of an RGB-D sensor is evaluated in terms of (1) the 3D positions of the body parts tracked by the sensor, (2) the 3D rotation angles at joints, and (3) the impact of the RGB-D sensor's accuracy on motion analysis. For the assessment, experimental studies were undertaken to collect motion capture datasets using an RGB-D sensor and a markerbased motion capture system, VICON, and to analyze errors as compared with the VICON used as the ground truth. As a test case, 25 trials of ascending and descending during ladder climbing were recorded simultaneously with both systems, and the resulting motion capture datasets (i.e., 3D skeleton models) were temporally and spatially synchronized for their comparison. Results: Through the comparative assessment, we found a discrepancy of 10.7 cm in the tracked locations of body parts, and a difference of 16.2 degrees in rotation angles. However, motion detection results show that the inaccuracy of an RGB-D sensor does not have a considerable effect on action recognition in the experiment. Conclusions: This paper thus provides insight into the accuracy of an RGB-D sensor on motion capture in various measures and directions of further research for the improvement of accuracy.

Hard-Hat Detection for Construction Safety Visualization

“…The authors investigated the performance of an RGB-D sensor to track a worker and or an object in terms of three parameters [18]: (1) 3D movement of a worker's body position, as tracked by the sensor; (2) 3D rotation angles of the body parts; and (3) analysis of the sensor's accuracy regarding body-part movement. These three datasets were gathered during experimental studies.…”

“…From the differences in pixels, the system determined whether the person was a worker. Han et al [18] used a vision-based motion detection to track unsafe working behavior of construction workers, using video camera images. A 3D model of a worker was developed using images from two different cameras.…”

A Stereo Vision-Based Approach to Marker-Less Motion Capture for On-Site Kinematic Modeling of Construction Worker Tasks

“…A 21 second motion sequence was simultaneously captured using Microsoft Kinect with iPi Recorder and Bumblebee with the system described in Section 2.3. For this experiment, recording was performed indoors for the purpose of evaluating the proposed approach's accuracy relative to Kinect, which has been shown previously to be incapable of tracking human motions in outdoor environments (Weerasinghe et al 2012;Han et al 2013b). During recording, a subject performed motions that included rotations of all major body segments along all three local axes.…”

“…Unfortunately, many of these systems utilize infrared-based RGB-D sensors (e.g. Microsoft Kinect) that are severely impeded by sunlight and ferromagnetic radiation and have limited operating range (Weerasinghe et al 2012;Han et al 2013b). Because these systems cannot be used outdoors, they are ineffective for monitoring worker tasks on construction sites.…”

Editorial visualization in engineering

“…Besides the development of visualization applications, emerging modeling technologies like Building Information Modeling (BIM) have also been presented by integrating visualization tools for multi-disciplinary design of construction (Kovacic et al 2013). Visualization devices like CCTV (Chen et al 2013) and RGB-D cameras (Han et al 2013) are used on building spaces and workers for monitoring and tracking purpose (Yabuki et al 2013). BIM experts utilized visual information for assisting decision-making processes regarding energy issues (Mhalas et al 2013).…”