CoRE: Constrained Robustness Evaluation of Machine Learning-Based Stability Assessment for Power Systems

Zhang, Zhenyong; Yau, David K. Y.

doi:10.1109/jas.2023.123252

Cited by 9 publications

(8 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, the integration of ML, DA, and uncertainty quantification has demonstrated promising outcomes in enhancing the accuracy and comprehension of models in diverse fields (Cheng et al., 2023). Our work builds on the growing literature describing ML‐based methods for learning and aiding the DA processes.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Four‐Dimensional Variational Constrained Neural Network‐Based Data Assimilation Method

Wang,

Ren,

Duan

et al. 2024

J Adv Model Earth Syst

View full text Add to dashboard Cite

Advances in data assimilation (DA) methods and the increasing amount of observations have continuously improved the accuracy of initial fields in numerical weather prediction during the last decades. Meanwhile, in order to effectively utilize the rapidly increasing data, Earth scientists must further improve DA methods. Recent studies have introduced machine learning (ML) methods to assist the DA process. In this paper, we explore the potential of a four‐dimensional variational (4DVar) constrained neural network (NN) method for accurate DA. Our NN is trained to approximate the solution of the variational problem, thereby avoiding the need for expensive online optimization when generating the initial fields. In the context that the full‐field system truths are unavailable, our approach embeds the system's kinetic features described by a series of analysis fields into the NN through a 4DVar‐form loss function. Numerical experiments on the Lorenz96 physical model demonstrate that our method can generate better initial fields than most traditional DA methods at a low computational cost, and is robust when assimilating observations with higher error outside of the distributions where it is trained. Furthermore, our NN‐based DA model is effective against Lorenz96 physical models with larger variable numbers. Our approach exemplifies how ML methods can be leveraged to improve both the efficiency and accuracy of DA techniques.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Further, Bocquet (2023) and Cheng et al. (2023) highlight the potential of ML and DA for improving the accuracy and efficiency of models in Earth sciences. In addition, the synergy between ML and DA has been highlighted by Boukabara et al.…”

Section: Introductionmentioning

confidence: 99%

A Four‐Dimensional Variational Constrained Neural Network‐Based Data Assimilation Method

Wang,

Ren,

Duan

et al. 2024

J Adv Model Earth Syst

View full text Add to dashboard Cite

show abstract

“…Reducing these model errors through improving the fundamental understanding of these processes and increasing the numerical resolution has been the subject of extensive past and ongoing efforts (Bonavita & Laloyaux, 2020; Danforth & Kalnay, 2008; Dunbar et al., 2021; Regayre et al., 2023). More recently, the rapid rise in the availability of high‐quality, frequent observations and algorithmic advances, particularly in data assimilation (DA) and machine learning (ML) (Cheng et al., 2023), can provide another promising direction for reducing such model errors and the resulting biases and uncertainties.…”

Section: Introductionmentioning

confidence: 99%

Interpretable Structural Model Error Discovery From Sparse Assimilation Increments Using Spectral Bias‐Reduced Neural Networks: A Quasi‐Geostrophic Turbulence Test Case

Mojgani,

Chattopadhyay,

Hassanzadeh

2024

J Adv Model Earth Syst

View full text Add to dashboard Cite

Earth system models suffer from various structural and parametric errors in their representation of nonlinear, multi‐scale processes, leading to uncertainties in their long‐term projections. The effects of many of these errors (particularly those due to fast physics) can be quantified in short‐term simulations, for example, as differences between the predicted and observed states (analysis increments). With the increase in the availability of high‐quality observations and simulations, learning nudging from these increments to correct model errors has become an active research area. However, most studies focus on using neural networks, which while powerful, are hard to interpret, are data‐hungry, and poorly generalize out‐of‐distribution. Here, we show the capabilities of Model Error Discovery with Interpretability and Data Assimilation (MEDIDA), a general, data‐efficient framework that uses sparsity‐promoting equation‐discovery techniques to learn model errors from analysis increments. Using two‐layer quasi‐geostrophic turbulence as the test case, MEDIDA is shown to successfully discover various linear and nonlinear structural/parametric errors when full observations are available. Discovery from spatially sparse observations is found to require highly accurate interpolation schemes. While NNs have shown success as interpolators in recent studies, here, they are found inadequate due to their inability to accurately represent small scales, a phenomenon known as spectral bias. We show that a general remedy, adding a random Fourier feature layer to the NN, resolves this issue enabling MEDIDA to successfully discover model errors from sparse observations. These promising results suggest that with further development, MEDIDA could be scaled up to models of the Earth system and real observations.

show abstract

“…Existing data-driven strategies to tackle such a challenge can be broadly categorized into physicsbased methods (e.g., proper orthogonal decomposition 4,[7][8][9][10] , dynamic mode decomposition [11][12][13][14] , Galerkin transforms 15,16 , and linear stochastic estimation 17,18 ) and neural networks (NNs) 1,2,[19][20][21][22][23][24][25][26][27][28][29] . The former, typically rooted in the linear approximation theory, suffers from the very limited ability of expressiveness and requires complete sensor measurement for full-field reconstruction.…”

Section: Introductionmentioning

confidence: 99%

Learning spatiotemporal dynamics with a pretrained generative model

Yang,

Li,

Han

et al. 2024

Preprint

View full text Add to dashboard Cite

Reconstructing spatiotemporal dynamics with sparse sensor measurement is an outstanding problem, commonly encountered in a wide spectrum of scientific and engineering applications. Such a problem is particularly challenging when the number and/or types of sensors (e.g., randomly placed) are extremely insufficient. Existing end-to-end learning models ordinarily suffer from the generalization issue for full-field reconstruction of spatiotemporal dynamics, especially in sparse data regimes typically seen in real-world applications. To this end, we propose a sparse-sensor-assisted score-based generative model (S3GM) to reconstruct and predict full-field spatiotemporal dynamics based on sparse measurements. Instead of learning directly the mapping between input and output pairs, an unconditioned generative model is firstly pretrained capturing the joint distribution of a vast group of pretraining data in a self-supervised manner, followed then by a sampling process conditioned on unseen sparse measurement. The efficacy of S3GM has been verified on multiple dynamical systems with various synthetic, real-world, and lab-test datasets (ranging from turbulent flow modeling to weather/climate forecasting). The results demonstrate the excellent performance of S3GM in zero-shot reconstruction and prediction of spatiotemporal dynamics even with high levels of data sparsity and noise. We find that S3GM exhibits high accuracy, generalizability, and robustness when handling different reconstruction tasks.

show abstract

CoRE: Constrained Robustness Evaluation of Machine Learning-Based Stability Assessment for Power Systems

Cited by 9 publications

References 13 publications

A Four‐Dimensional Variational Constrained Neural Network‐Based Data Assimilation Method

A Four‐Dimensional Variational Constrained Neural Network‐Based Data Assimilation Method

Interpretable Structural Model Error Discovery From Sparse Assimilation Increments Using Spectral Bias‐Reduced Neural Networks: A Quasi‐Geostrophic Turbulence Test Case

Learning spatiotemporal dynamics with a pretrained generative model

Contact Info

Product

Resources

About