Reducing the effect of sample bias for small data sets with double‐weighted support vector transfer regression

Abstract

Small data sets are an extremely challenging problem in the machine learning (ML) realm, and in specific, in regression scenarios, as the lack of relevant data can lead to ML models that have large bias. However, there are many applications for which a purely data‐driven procedure would be advantageous, but a large amount of data are not available. This article proposes a novel regression‐based transfer learning (TL) model to address this challenge, where TL is defined as knowledge transfer from a large, relevant data set (source domain data) to a small data set (target domain data). The proposed TL model is termed double‐weighted support vector transfer regression (DW‐SVTR), which couples least squares support vector machines for regression (LS‐SVMR) with two weight functions. The first weight function uses kernel mean matching (KMM) to reweight the source domain data such that the mean values of the source and target domain data in a reproduced kernel Hilbert space (RKHS) are close. In this way, the source domain data points relevant to the target domain points have a larger weight than irrelevant source domain points. The second weight is a function of estimated residuals, which aims to further reduce the negative interference of irrelevant source domain points. The proposed approach is assessed and validated via simulated data and by enhanced shear strength prediction of nonductile columns based on limited availability of nonductile column data. Specifically, the results for the latter show that the proposed DW‐SVTR can reduce the root mean square error (RMSE) by 34% and enhance the coefficient of determination (R2) by 229%. These numerical results demonstrate that the DW‐SVTR significantly reduces the effect of small sample bias and improves prediction performance compared to standard ML methods.

Using machine learning to analyze and predict construction task productivity

“…Specifically, productivity studies have benefited from the application of ML because these techniques serve to determine the relationship between the influencing factors and productivity rates and the complexity of the combined effects between factors (Boussabaine & Kirkham, 2008; Chao & Skibniewski, 1994; Franco & Santurro, 2020; Karim & Adeli, 1999). ML has been used in the design stage (As et al., 2018; Huang & Zheng, 2018; Luo & Paal, 2020; Rodrigues et al., 2017; Valikhani et al., 2020), construction stage (Bai et al., 2019; Poh et al., 2018; Xiong & Huber, 2010; Yu et al., 2019), and the operation and maintenance stage (Farrar & Worden, 2012; Maeda et al., 2020; Okazaki et al., 2020; Yang & Su, 2008; Zhang et al., 2019). ML has also been used in civil engineering applications such as damage assessment of buildings (Cheng et al., 2021), flood warning (Dong et al., 2020), risk assessment of infrastructure systems (Tomar & Burton, 2021).…”

Cost-sensitive regression learning on small dataset through intra-cluster product favoured feature selection

“…Other studies have focussed on the realm of regression learning, including diffusion neural network, data trend estimation techniques, omega-trend-diffusion techniques, central location data tracking methods, structure-based data transformation methods, domain knowledge, and transfer learning methods (Guo et al, 2017;Huang & Moraga, 2004;Li, Chang, et al, 2012;Luo & Paal, 2021). However, few studies have paid attention to cost-sensitive regression learning on small datasets.…”

“…Dimensionality reduction Baumgartner et al (2004), Kursa and Rudnicki (2010), Chandrashekar and Sahin (2014), Micallef et al (2017), Mishra and Singh (2020). Learning methods Huang and Moraga (2004), Chen et al (2005), Mao et al (2006), Li and Liu (2009), Li, Chang, et al (2012), Guo et al (2017), Luo and Paal (2021).…”

Data-driven seismic response prediction of structural components

“…With more and more experimental data available, machine learning (ML) techniques have been successfully applied in many engineering and science domains (Adeli, 2001; Reich, 1997). For example, support vector machines for regression (SVMR) (Vapnik, 1995) and their extension version, least squares-SVMR (LS-SVMR) (Suykens et al, 2002), have been used to predict the shear strength of RC deep beams (Chou et al, 2015; Pal and Deswal, 2011), the punching shear capacity of fiber-reinforced polymer (FRP) RC slabs (Vu and Hoang, 2016), and the backbone curve, lateral strength, and drift capacity of RC columns (Luo and Paal, 2018, 2019, 2021a, 2021b, 2021c). Artificial neural networks (ANNs) and genetic programming (GP) have been used to predict the punching shear strength of RC slabs (Gandomi and Roke, 2015) and identify the failure mode of RC bridge columns (Mangalathu and Jeon, 2019).…”