Multitask Learning Method for Detecting the Visual Focus of Attention of Construction Workers

Cai, Jiannan; Liu, Yang; Zhang, Yuxi; Li, Shuai; Cai, Hubo

doi:10.1061/(asce)co.1943-7862.0002071

Cited by 10 publications

(10 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address this issue, the authors modified the source code of YOLO v5 to remove horizontal flipping in the mosaic data enhancement when training the pose and orientation dataset. Table 5 presents the relevant performance metrics, and the mAP achieved a competitive accuracy of 0.898 using the YOLO v5 model, which is comparable to similar studies [43]. Figure 15 demonstrates accurate recognition of the pose and orientation of individuals in most cases.…”

Section: Yolo V5-based Model For Estimating Workers' Pose and Orienta...supporting

confidence: 69%

“…Orientation estimation, on the other hand, is a less researched area. Cai et al [43] used Faster R-CNN to detect the head and body orientation of construction workers and then applied a multi-task learning network to assess the direction of visual attention of construction workers. Person re-identification technology is a popular direction of computer vision in recent years [44], but the technology is not much used in the fields of engineering and construction.…”

Section: Computer Vision Technology and Engineering Applicationsmentioning

confidence: 99%

“…These studies categorized an individual's orientation into eight categories: East, Northeast, North, Northwest, West, Southwest, South, and Southeast. However, according to Cai et al [43], accurately assessing the orientation of a human body is challenging due to the low resolution of most construction surveillance video images. To address this issue, Cai et al categorized both the body orientation and the head orientation into four categories: East, South, West, and North.…”

Section: Dataset For Estimation Of Posture and Orientationmentioning

confidence: 99%

See 2 more Smart Citations

Construction Activity Recognition Method Based on Object Detection, Attention Orientation Estimation, and Person Re-Identification

Li,

Zhao,

Kong

et al. 2024

Buildings

View full text Add to dashboard Cite

Recognition and classification for construction activities help to monitor and manage construction workers. Deep learning and computer vision technologies have addressed many limitations of traditional manual methods in complex construction environments. However, distinguishing different workers and establishing a clear recognition logic remain challenging. To address these issues, we propose a novel construction activity recognition method that integrates multiple deep learning algorithms. To complete this research, we created three datasets: 727 images for construction entities, 2546 for posture and orientation estimation, and 5455 for worker re-identification. First, a YOLO v5-based model is trained for worker posture and orientation detection. A person re-identification algorithm is then introduced to distinguish workers by tracking their coordinates, body and head orientations, and postures over time, then estimating their attention direction. Additionally, a YOLO v5-based object detection model is developed to identify ten common construction entity objects. The worker’s activity is determined by combining their attentional orientation, positional information, and interaction with detected construction entities. Ten video clips are selected for testing, and a total of 745 instances of workers are detected, achieving an accuracy rate of 88.5%. With further refinement, this method shows promise for a broader application in construction activity recognition, enhancing site management efficiency.

show abstract

Section: Yolo V5-based Model For Estimating Workers' Pose and Orienta...supporting

confidence: 69%

Section: Computer Vision Technology and Engineering Applicationsmentioning

confidence: 99%

Section: Dataset For Estimation Of Posture and Orientationmentioning

confidence: 99%

See 1 more Smart Citation

Construction Activity Recognition Method Based on Object Detection, Attention Orientation Estimation, and Person Re-Identification

Li,

Zhao,

Kong

et al. 2024

Buildings

View full text Add to dashboard Cite

show abstract

“…It is only required to read the information in data images and videos through a computer, and then manage the site through analysis [42,43]. Object detection through computer vision (CV) in construction sites primarily focuses on monitoring workers, identifying unsafe behaviors, and ensuring PPE compliance [36,[44][45][46]. Detection methods are broadly categorized into two-stage and single-stage methods [47].…”

Section: B Ppe Detection Technology Based On Deep Learningmentioning

confidence: 99%

Personal Protective Equipment Detection for Construction Workers: A Novel Dataset and Enhanced YOLOv5 Approach

Yipeng,

Junwu

2024

IEEE Access

View full text Add to dashboard Cite

Current research on personal protective equipment (PPE) detection has mainly focused on hard hats, overlooking the detection of reflective clothing. Therefore, this study aims to address this research gap comprehensively. We achieve this by creating a novel dataset using semi-automatic labeling techniques and enhancing the YOLOv5 model. The dataset consists of four categories, assessing the presence of hard hats and reflective clothing. Additionally, we introduce an attention mechanism and an improved loss function to tackle challenges related to detecting reflective clothing and overlapping detection frames. Through extensive multi-model comparison experiments, our improved model, AL-YOLOv5, outperforms the baseline model with remarkable advancements of 0.9 AP in the data-limited category and 0.4 mAP overall. Notably, our improved model shows substantial progress in detecting reflective clothing, significantly reducing false detections, and improving overlapping bounding frames. In conclusion, this study contributes to PPE detection accuracy through a novel dataset and an improved model.

show abstract

“…Over the years, several civil engineering works have used MTL to improve performance, training time, and inference time. Most recently, Cai et al (2021) estimated the visual focus of attention of construction workers through noisy low-resolution photographs. They reported that formulating their solution as an MTL increased prediction accuracies on the two harder categories (body orientation and head yaw) by 2% each.…”

Section: Multitask Learningmentioning

confidence: 99%

Deep Learning–Based Building Attribute Estimation from Google Street View Images for Flood Risk Assessment Using Feature Fusion and Task Relation Encoding

Chen

Subedi

Jahanshahi

et al. 2022

J. Comput. Civ. Eng.

View full text Add to dashboard Cite

Floods are the most common and damaging natural disaster worldwide in terms of both economic losses and human casualties. Currently, policymakers rely on data collected through labor-intensive and, consequently, expensive street-level surveys to assess flood risks. We propose a laborless and financially feasible alternative: a framework that can effectively and efficiently collect building attribute data without manual street surveys. By utilizing deep learning, the proposed framework analyzes Google Street View (GSV) images to estimate multiple attributes of buildings simultaneously-including foundation height, foundation type, building type, and number of stories-that are necessary for assessing flood risks. The proposed framework achieves a 0.177-m mean absolute error (MAE) for foundation height estimation and classification F1 scores of 77.96% for foundation type, 83.12% for building type, and 94.60% for building stories, and requires less than five days to predict the attributes of 0.8 million buildings in coastal Louisiana.

show abstract

Multitask Learning Method for Detecting the Visual Focus of Attention of Construction Workers

Cited by 10 publications

References 42 publications

Construction Activity Recognition Method Based on Object Detection, Attention Orientation Estimation, and Person Re-Identification

Construction Activity Recognition Method Based on Object Detection, Attention Orientation Estimation, and Person Re-Identification

Personal Protective Equipment Detection for Construction Workers: A Novel Dataset and Enhanced YOLOv5 Approach

Deep Learning–Based Building Attribute Estimation from Google Street View Images for Flood Risk Assessment Using Feature Fusion and Task Relation Encoding

Contact Info

Product

Resources

About