Practical benchmarking of statistical and machine learning models for predicting the condition of sewer pipes in Berlin, Germany

Abstract

Deterioration models can be successfully deployed only if decision-makers trust the modelling outcomes and are aware of model uncertainties. Our study aims to address this issue by developing a set of clearly understandable metrics to assess the performance of sewer deterioration models from an end-user perspective. The developed metrics are used to benchmark the performance of a statistical model, namely, GompitZ based on survival analysis and Markov-chains, and a machine learning model, namely, Random Forest, an ensemble learning method based on decision trees. The models have been trained with the extensive CCTV dataset of the sewer network of Berlin, Germany (115,258 inspections). At network level, both models give satisfactory outcomes with deviations between predicted and inspected condition distributions below 5%. At pipe level, the statistical model does not perform better than a simple random model, which attributes randomly a condition class to each inspected pipe, whereas the machine learning model provides satisfying performance. 66.7% of the pipes inspected in bad condition have been predicted correctly. The machine learning approach shows a strong potential for supporting operators in the identification of pipes in critical condition for inspection programs whereas the statistical approach is more adapted to support strategic rehabilitation planning.

Sewer asset management – state of the art and research needs

“…They indicated that survival analysis and Markov models outperform a simple random model for predicting the condition distribution of the network, especially in the case of low data availability. Caradot et al (2018) also showed that statistical models have a clear advantage against machine learning models at the network level when extrapolating beyond the observation window. Many studies (see Table 3) assessed model performance at the pipe level.…”

“…Few studies evaluated the performance of deterioration models to simulate the condition distribution of the network (Caradot et al 2017(Caradot et al , 2018Duchesne et al 2013;Hernández et al 2018;Ugarelli et al 2013). They indicated that survival analysis and Markov models outperform a simple random model for predicting the condition distribution of the network, especially in the case of low data availability.…”

Measurement of Wastewater Discharge in Sewer Pipes Using Image Analysis

“…Several researchers [1][2][3][4] have conducted studies on sewerage using the above computational methods to diagnose pipe conditions; they evaluated pipe defects by analyzing the images of the sewer pipes. Kumar et al [5] detected and classified pipe defects from images.…”

Combining recorded failures and expert opinion in the development of ANN pipe failure prediction models

“…These models have two primary objectives: estimating long-term failure rates associated with a given management strategy for budgetary planning purposes and identifying pipes that are likely to fail in the near future for prioritization (Lin & Yuan, 2019;Yuan, 2016). Recently, several independent research groups have observed that failure prediction models based on machine learning algorithms (MLAs) appear to perform better in identifying critical pipes in the short term and models based on survival analysis are better able to predict long-term failure trends (Caradot et al, 2018;Snider & McBean, 2019;Tscheikner-Gratl et al, 2019). Snider and McBean (2019) attributed this observation to the ability of survival analysis based models to correct for data that is left-truncated (i.e., failure data is only available from the point when recording began) or right-censored (i.e., future failures are unknown).…”