Unsupervised data normalization for continuous dynamic monitoring by an innovative hybrid feature weighting-selection algorithm and natural nearest neighbor searching

Abstract

Continuous dynamic monitoring brings an important opportunity to evaluate the health and integrity of civil structures in a long-term manner. However, high dimensionality and sparsity of data caused by long-term monitoring and negative influences of environmental and/or operational variability are major challenges in this process. To address these important issues, this article proposes an innovative unsupervised data normalization method based on a novel hybrid feature weighting-selection algorithm and the idea of natural nearest neighbor (NN) searching emanated from the theory of mutual friendships in human societies. The proposed hybrid algorithm is a combination of global feature weighting with a new weighting measure and local feature selection. For this algorithm, this article leverages the natural NN searching that seeks to find adequate NNs automatically. The main objective of the proposed method is to remove the environmental and/or operational effects and provide normalized weighted features for reliable continuous dynamic monitoring. Using such features, an anomaly detector based on the Mahalanobis-squared distance is developed to assess and detect structural damage. The key innovations of this paper contain proposing a fully nonparametric unsupervised learning technique in two parts of data normalization and anomaly detection and developing a novel hybrid algorithm for removing the environmental and/or operational variations. Long-term dynamic features (modal frequencies) of a three-span box-girder concrete bridge (Z24 Bridge) and a long-span concrete arch bridge (Infante Dom Henrique Bridge) are considered to verify the proposed technique with several comparisons. Results indicate that this technique is successful and reliable in mitigating the environmental and/or operational effects and notifying accurate structural states.

Elimination of Thermal Effects from Limited Structural Displacements Based on Remote Sensing by Machine Learning Techniques

“…Sarmadi et al [39] took advantage of the idea of unsupervised nearest neighbor searching and semi-parametric extreme value theory to propose a novel data self-clustering method for eliminating the environmental variations from modal frequencies in long-term monitoring. Sarmadi et al [40] proposed a novel unsupervised data normalization method based on a hybrid feature weighting-selection algorithm with the aid of a new nearest neighbor search for removing the environmental effects from modal frequencies of two full-scale bridges.…”

A Parsimonious Yet Robust Regression Model for Predicting Limited Structural Responses of Remote Sensing

“…The choice of an appropriate sensing technology and of the measurement of the structural response to different natural or man-made excitation sources is critical to provide data sensitive to the structural state. The process of data analytics is often conducted through data cleaning, compression, fusion [10], data augmentation [11], data prediction [12], data normalization [13], and feature extraction [14]. Different machine learning algorithms within the realms of unsupervised learning [15][16][17][18] and supervised learning [19] can be adopted for decision-making about whether the bridge has suffered damage or can still operate normally.…”

Regression Tree Ensemble to Forecast Thermally Induced Responses of Long-Span Bridges

“…This is because spatial and temporal temperature variations cause thermal loads, unpredictable internal stresses and forces in elements, and also changes in the boundary conditions, particularly for large-scale bridges, all of which lead to damages such as cracks in concrete and yielding of steel elements [11]. Therefore, it is essential to regularly monitor structures like long-span bridges under varying ambient temperature effects to remove such effects [12][13][14][15].…”