Discovering and forecasting extreme events via active learning in neural operators

Abstract

Extreme events in society and nature, such as pandemic spikes or rogue waves, can have catastrophic consequences. Characterizing extremes is difficult as they occur rarely, arise from seemingly benign conditions, and belong to complex and often unknown infinite-dimensional systems. Such challenges render attempts at characterizing them as moot. We address each of these difficulties by combining novel training schemes in Bayesian experimental design (BED) with an ensemble of deep neural operators (DNOs). This model-agnostic framework pairs a BED scheme that actively selects data for quantifying extreme events with an ensemble of DNOs that approximate infinite-dimensional nonlinear operators.We find that not only does this framework clearly beat Gaussian processes (GPs) but that 1) shallow ensembles of just two members perform best; 2) extremes are uncovered regardless of the state of initial data (i.e. with or without extremes); 3) our method eliminates "double-descent" phenomena; 4) the use of batches of suboptimal acquisition points compared to step-by-step global optima does not hinder BED performance; and 5) Monte Carlo acquisition outperforms standard minimizers in high-dimensions.Together these conclusions form the foundation of an AI-assisted experimental infrastructure that can efficiently infer and pinpoint critical situations across many domains, from physical to societal systems.

In Pursuit of the Exceptional: Research Directions for Machine Learning in Chemical and Materials Science

“…131 In the context of Bayesian optimization type strategies, an appropriate approach is the output-weighted optimal sampling introduced by Blanchard and Sapsis and co-workers, 133−135 which has been recently applied to extreme event discovery in epidemiological models, rogue waves, and structure mechanics. 136 V.B. When data are limited, qualitative prediction of direction to the goal is more valuable than (interpolative) accuracy.…”

“…Alternative problem formulations, such as the Max- k -arm bandit model, better align with the goals of scientific discovery, as demonstrated with in silico numerical experiments of exploring molecular SMILES strings to maximize the boiling point and other thermophysical properties described by an empirical proxy . In the context of Bayesian optimization type strategies, an appropriate approach is the output-weighted optimal sampling introduced by Blanchard and Sapsis and co-workers, − which has been recently applied to extreme event discovery in epidemiological models, rogue waves, and structure mechanics …”

Neural Operators for Bypassing Gain and Control Computations in PDE Backstepping

“…NOMAD [68] extends the linear reconstructor map in DeepONet to a nonlinear map that is capable of learning on nonlinear submanifolds in function spaces. There have been many more extensions to the neural operator architectures omitted here as they are usually designed around domain-specific enhancements [83] [49] [63]. Another line of work, called physics-informed neural networks (PINNs) [36], [66], which can be used as generic solvers of PDEs by adding physics constraint loss to neural networks.…”

Statistical modeling of fully nonlinear hydrodynamic loads on offshore wind turbine monopile foundations using wave episodes and targeted CFD simulations through active sampling

“…They support various analyses, including sensitivity analysis, optimization, uncertainty quantification, and statistical reconstruction for defining ULS and FLS. Surrogate modeling, such as with Gaussian process regression (GPR), 35,36 additionally allows the deployment of techniques from the optimal experimental design literature, [37][38][39] where initial simulation or experimental results are used to improve subsequent designs. [40][41][42][43][44] In addition, scientists have addressed the question of experimental design for these studies-what waves to simulate-with a number of methods, including stochastic wavegroups, 45,46 critical wavegroups, [47][48][49][50] equivalent waves, 51 reduced order wavegroups, 40,52,53 and Karhunen-Loève (KL) wave episodes.…”

“…They support various analyses, including sensitivity analysis, optimization, uncertainty quantification, and statistical reconstruction for defining ULS and FLS. Surrogate modeling, such as with Gaussian process regression (GPR), 35,36 additionally allows the deployment of techniques from the optimal experimental design literature, 37–39 where initial simulation or experimental results are used to improve subsequent designs 40–44 …”