Detecting Construction Equipment Using a Region-Based Fully Convolutional Network and Transfer Learning

Kim, Hongjo; Kim, Hyoungkwan; Hong, Yong Won; Byun, Hyeran

doi:10.1061/(asce)cp.1943-5487.0000731

Cited by 189 publications

(85 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The proposed method was performed on the Intel i7-6700 CPU and the GTX1080 8GB GPU with the Ubuntu 16.04 operating system. The R-FCN model [9] pretrained with the ImageNet data was re-trained using the AIMdataset [10] and the tunnel image data. The length of the tunnel video for the test was 8 h 11 m. The video resolution was 720 × 480.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…To detect the target objects in tunnel CCTV image data, a region-based fully convolutional networks (R-FCN) proposed by Dai, et al [9] is utilized. Using R-FCN, construction equipment such excavators and dump trucks can be detected with the high accuracy at a short processing time [10]. R-FCN comprises three main modules for object detection, which are the feature extraction layers, region proposal network, and the position-sensitive score maps for determining the object class and location in image data.…”

Section: Vision-based Context Reasoningmentioning

confidence: 99%

See 1 more Smart Citation

Integration of Imaging and Simulation for Earthmoving Productivity Analysis

Kim

Bang

Jeong

et al. 2018

Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC)

Self Cite

View full text Add to dashboard Cite

Earthwork productivity varies depending on a unique geologic condition, types of earthwork equipment, and an equipment allocation plan. For this reason, it is difficult to accurately estimate the productivity of an earthwork. To address this issue, this paper develops an imaging-to-simulation method in which a real jobsite data is automatically collected and used for analyzing the earthwork productivity. Object existence and its location in image data are identified by convolutional networks, and they are used to infer the earthwork context. The context information is transformed into the simulation input by the context reasoning processes. A productivity report is produced by using the WebCYCLONE simulation. The developed method was tested in a tunnel construction site, providing a new equipment allocation plan, which minimize the cost and time compared with the original plan.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Vision-based Context Reasoningmentioning

confidence: 99%

Integration of Imaging and Simulation for Earthmoving Productivity Analysis

Kim

Bang

Jeong

et al. 2018

Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Deep learning, because of its flexibility with large databases and its capability to handle a large number of classifications, has received significant attention from construction researchers [15][16][17][18][19][20][21][22][23][24][25][26]. For example, with safety applications, researchers have used deep learning to identify non-certified workers on construction sites, in order to prevent safety hazards [15]; to identify workers, equipment, and materials [16][17][18]; to identify unsafe worker behavior, in order to prevent accidents [19][20][21][22]; and to detect guardrails [23] and cracks [24][25][26]. However, all of these studies are related to computer vision, while the authors in this study intend to automate the prediction of scaffold safety conditions using numerical strain measurement values.…”

Section: Application Of Deep Learningmentioning

confidence: 99%

Multi-Level-Phase Deep Learning Using Divide-and-Conquer for Scaffolding Safety

Sakhakarmi

Park

2020

IJERPH

View full text Add to dashboard Cite

A traditional structural analysis of scaffolding structures requires loading conditions that are only possible during design, but not in operation. Thus, this study proposes a method that can be used during operation to make an automated safety prediction for scaffolds. It implements a divide-and-conquer technique with deep learning. As a test scaffolding, a four-bay, three-story scaffold model was used. Analysis of the model led to 1411 unique safety cases for the model. To apply deep learning, a test simulation generated 1,540,000 datasets for pre-training, and an additional 141,100 datasets for testing purposes. The cases were then sub-divided into 18 categories based on failure modes at both global and local levels, along with a combination of member failures. Accordingly, the divide-and-conquer technique was applied to the 18 categories, each of which were pre-trained by a neural network. For the test datasets, the overall accuracy was 99%. The prediction model showed that 82.78% of the 1411 safety cases showed 100% accuracy for the test datasets, which contributed to the high accuracy. In addition, the higher values of precision, recall, and F1 score for the majority of the safety cases indicate good performance of the model, and a significant improvement compared with past research conducted on simpler cases. Specifically, the method demonstrated improved performance with respect to accuracy and the number of classifications. Thus, the results suggest that the methodology could be reliably applied for the safety assessment of scaffolding systems that are more complex than systems tested in past studies. Furthermore, the implemented methodology can easily be replicated for other classification problems.

show abstract

“…Convolutional neural networks (CNNs) have been highlighted in many image‐based problems. An extensive number of SHM applications of CNNs focus on vision‐based surface defect detection and image recognition for construction safety . Intuitively, CNNs imitate the sensing functions of the animal visual cortex (individual cortical neurons respond to stimuli only in a restricted region of the visual field) by gathering information from neighboring inputs to form subfeatures in the filters.…”

Section: Introductionmentioning

confidence: 99%

Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring

Tang

Chen

Bao

et al. 2018

Struct Control Health Monit

274

156

View full text Add to dashboard Cite

Structural health monitoring (SHM) is used worldwide for managing and maintaining civil infrastructures. SHM systems have produced huge amounts of data, but the effective monitoring, mining, and utilization of this data still need in-depth study. SHM data generally includes multiple types of anomalies caused by sensor faults or system malfunctions that can disturb structural analysis and assessment. In the routine data pre-processing, multiple signal processing techniques are required to detect the anomalies, respectively, which is inefficient. The large variations of extracted features from massive SHM data make the data anomaly detection techniques prone to be over-processed or under-processed. Even with expert intervention, the parameter tuning, associated with multiple data preprocessing methods, is still a challenge, making the procedure expensive and inefficient. In addition, one data anomaly detection technique frequently mis-detects other types of anomaly. In this work, we focus on the anomaly detection in the stage of data pre-processing that little work has been done based on the real-world continuous SHM data with multiclass anomalies. We proposed a novel data anomaly detection method based on a convolutional neural network (CNN) that imitates human vision and decision making. First, we split raw time series data into sections, and visualized the data in time and frequency domain, respectively. Then each section's images were stacked as a single dual-channel image and labeled according to graphical features (multi-2D image space expression). Second, a CNN was designed and trained for data anomaly classification, during which the descriptions and representations of the anomalies' features were learned by convolution. To validate our work, we considered the effects of balanced and imbalanced training sets and training ratios on actual acceleration data from an SHM system for a long-span cable-stayed bridge. The results show that our method could detect the multipattern anomalies of SHM data efficiently with high accuracy. The proposed dual-information CNN-based design makes this detection process readily scalable, faster, and more accurate, thereby providing a novel perspective with strong potential for SHM data preprocessing. KEYWORDScomputer vision, convolutional neural network (CNN), data anomaly detection, long-span bridge, structural health monitoring (SHM) | INTRODUCTIONStructural health monitoring (SHM) is used worldwide to manage and maintain civil infrastructure systems by evaluating their structural loads, responses, and real-time performance, and by predicting the future behavior of structures of all types. 1-4 SHM has produced huge amounts of data. For example, the SHM system used for the Sutong Bridge in China, which has 785 sensors, produces 2.5 TB of data each year. The effective mining and utilization of SHM data is an important topic that needs in-depth study. However, researchers have faced challenges because the SHM data generally include multiple types of anomalies caused by sensor fa...

show abstract

Detecting Construction Equipment Using a Region-Based Fully Convolutional Network and Transfer Learning

Cited by 189 publications

References 67 publications

Integration of Imaging and Simulation for Earthmoving Productivity Analysis

Integration of Imaging and Simulation for Earthmoving Productivity Analysis

Multi-Level-Phase Deep Learning Using Divide-and-Conquer for Scaffolding Safety

Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring

Contact Info

Product

Resources

About