Developing machine learning tools for long-lead heavy precipitation prediction with multi-sensor data

Abstract

A large number of extreme floods were closely related to heavy precipitation which lasted for several days or weeks. Long-lead prediction of extreme precipitation, i.e., prediction of 6-15 days ahead of time, is important for understanding the prognostic forecasting potential of many natural disasters, such as floods. Yet, long-lead flood forecasting is a challenging task due to the cascaded uncertainty with prediction errors from measurements to modeling, which makes the current physics-based numerical simulation models extremely complex and inaccurate. In this paper, we formulate the modeling work as a machine learning problem and introduce a complementary data mining framework for heavy precipitation prediction. Heavy precipitation that may lead to extreme floods is a rare event. Long-lead prediction requires the corresponding feature space to be sampled from extremely high spatio-temporal dimensions. Such a complexity makes long-lead heavy precipitation prediction a high dimensional and imbalanced machine learning problem. In this work, we firstly define the extreme precipitation and non-extreme precipitation clusters and then design the Nearest-Sample Choosing method to handle the imbalanced data sets. We introduce streaming feature selection and subspace learning to extract the most relevant features from high dimensional data. We evaluate the machine learning tools using historical flood data collected in the State of Iowa, the United States and associated hydrometeorological variables from 1948 to 2010.

An evaluation of big data analytics in feature selection for long-lead extreme floods forecasting

“…Experiment 1: Four online streaming feature selection methods + original data + KNN [13]. The aim of this experiment is to check the effect of four online streaming feature selection methods on imbalanced data.…”

“…The result of experiments, The classifier of 1-knn[13] is used to build the prediction model for both the validation and evaluation processes.…”

Survey of computational intelligence as basis to big flood management: challenges, research directions and future work

“…Using TSDM and chaos theory in river flood prediction (Lobbrecht & Solomatine, 2002) ANN Using machine learning approaches (ANN and fuzzy adaptive system) in flood forecasting ANN Developing an ANN-based model with a simple structure and ample accuracy to predict the waste generation amount (Duncan et al, 2013) A time-lagged ANN Using ANN to predict flooding in a real-time manner relying on weather radar and rain gauge rainfall information (Yazdi & Neyshabouri, 2014) MOGA and ANN Presenting an adaptive meta model for flood forecasting using a combination of hydrodynamic model, MOGA and ANN (Kia et al, 2012) ANN together with GIS Developing a model for flooding using several flood causative factors employing an ANN method and GIS (Osanai, Shimizu, Kuramoto, Kojima, & Noro, 2010) ANN-RBF Developing a system to pre-identify the debris flows and slope failures based on rainfall indices using a RBF network (Baum & Godt, 2010) ANN-RBF Presenting a pre-identification system to identify the debris flows and rainfall-induced shallow landslides (Holz, Hildebrandt, & Weber, 2006) ANN based on DM Study of using information potential and communication technology (ICT) in a flood management system (Schnebele, 2013) ML classification Proposing a new method for flood assessment based on remote sensing and ML (Di et al, 2015) ML Proposing a ML approach for long lead flood forecasting by applying extreme precipitation and non-extreme precipitation definition (Napolitano, See, Calvo, Savi, & Heppenstall, 2010) ANN Presenting NN model for hourly water level forecasting using an adaptive, conceptual Tevere Flood Forecasting model and a data-driven approach using the applied TNN model (Rahim & Akif, 2015) ANN Proposing an optimized ANN model to predict the runoff and sedimentation yield (Nagy, Watanabe, & Hirano, 2002) ANN Prediction of sediment load concentration in rivers using an ANN model (Pyayt, Mokhov, Lang, Krzhizhanovskaya, & Meijer, 2011) AI component with neural clouds…”

“…Simulating physical-based numerical models of these phenomena are incredibly complex and inaccurate, as it is of primary importance to predict natural disasters 6-15 days ahead of time. Di et al (2015) used an ANN combined with a nearest sample to handle imbalanced data sets. The maximum accuracy obtained reached 60.5% for predictions made at a suitable time in advance.…”

Machine Learning of Key Variables Impacting Extreme Precipitation in Various Regions of the Contiguous United States

“…Fan & Li, 2006; Johnstone & Titterington, 2009). Advanced machine learning (ML) techniques bring hope to tackle the problem and have been employed to advance the characterization and modeling of extreme events, identify important variables, and provide a predictive or mechanistic understanding of extremes (Cramer et al., 2017; Di et al., 2015; Knighton et al., 2019; M. J. Molina et al., 2021). One challenge toward a comprehensive mechanistic understanding with ML is the mixed types of data with different spatial and temporal variations between the response and explanatory variables.…”