Data Augmentation for Deep-Learning-Based Multiclass Structural Damage Detection Using Limited Information

Abstract

The deterioration of infrastructure’s health has become more predominant on a global scale during the 21st century. Aging infrastructure as well as those structures damaged by natural disasters have prompted the research community to improve state-of-the-art methodologies for conducting Structural Health Monitoring (SHM). The necessity for efficient SHM arises from the hazards damaged infrastructure imposes, often resulting in structural collapse, leading to economic loss and human fatalities. Furthermore, day-to-day operations in these affected areas are limited until an inspection is performed to assess the level of damage experienced by the structure and the required rehabilitation determined. However, human-based inspections are often labor-intensive, inefficient, subjective, and restricted to accessible site locations, which ultimately negatively impact our ability to collect large amounts of data from inspection sites. Though Deep-Learning (DL) methods have been heavily explored in the past decade to rectify the limitations of traditional methods and automate structural inspection, data scarcity continues to remain prevalent within the field of SHM. The absence of sufficiently large, balanced, and generalized databases to train DL-based models often results in inaccurate and biased damage predictions. Recently, Generative Adversarial Networks (GANs) have received attention from the SHM community as a data augmentation tool by which a training dataset can be expanded to improve the damage classification. However, there are no existing studies within the SHM field which investigate the performance of DL-based multiclass damage identification using synthetic data generated from GANs. Therefore, this paper investigates the performance of a convolutional neural network architecture using synthetic images generated from a GAN for multiclass damage detection of concrete surfaces. Through this study, it was determined the average classification performance of the proposed CNN on hybrid datasets decreased by 10.6% and 7.4% for validation and testing datasets when compared to the same model trained entirely on real samples. Moreover, each model’s performance decreased on average by 1.6% when comparing a singular model trained with real samples and the same model trained with both real and synthetic samples for a given training configuration. The correlation between classification accuracy and the amount and diversity of synthetic data used for data augmentation is quantified and the effect of using limited data to train existing GAN architectures is investigated. It was observed that the diversity of the samples decreases and correlation increases with the increase in the number of synthetic samples.

From data to insight, enhancing structural health monitoring using physics-informed machine learning and advanced data collection methods

“…In DL, the learners show reliable performance for noisy datasets and hence raw sensor data can be directly fed into the classifier without pre-processing [69,70]. However, the major limitation of DL is its requirement for a large number of training data which can be somehow obtained through different augmentation techniques [71][72][73][74][75]. However, this approach will not be viable if the training dataset is too short to represent the basic distinguishable characteristics in the data.…”

Image-to-Image Translation-Based Structural Damage Data Augmentation for Infrastructure Inspection Using Unmanned Aerial Vehicle

“…GANbased data augmentation proves valuable in enhancing model performance across various fields, especially in situations where data acquisition is challenging. Dunphy et al [19] validated the use of synthetically generated images from GANs for multi-class damage detection on concrete surfaces. Their research indicated that the average classification performance for hybrid datasets decreased by approximately 10.6% and 7.4% for validation and testing datasets, respectively, when compared to models trained solely on real samples.…”

“…While common data augmentation methods like random flipping, rotating, and cropping have demonstrated their ability to bolster the resilience of trained DNNs, they possess limitations in encapsulating the authentic diversity seen in real-world structures. This includes the variations in building attributes such as size, shape, and color, consequently curbing these methods' efficacy [18][19][20]. In response to this issue, researchers have introduced innovative data augmentation strategies that leverage synthetic images-either generated by computers or data-driven [21,22].…”

Industrial Anomaly Detection with Skip Autoencoder and Deep Feature Extractor

“…However, the training of CNNs requires many defect samples and necessitates additional manpower for defect labeling, which makes it difficult to import deep learning technology into actual production lines. Although many data augmentation methods have been proposed thus far [ 11 , 12 , 13 , 14 ], the application of CNNs in industrial testing remains limited.…”