Machine-learning parameter tracking with partial state observation

Zhai, Zheng-Meng; Moradi, Mohammadamin; Glaz, Bryan; Haile, Mulugeta; Lai, Ying-Cheng

doi:10.1103/physrevresearch.6.013196

Phys. Rev. Research

2024

DOI: 10.1103/physrevresearch.6.013196

|View full text |Cite

Machine-learning parameter tracking with partial state observation

Zheng-Meng Zhai,

Mohammadamin Moradi,

Bryan Glaz

et al.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article2

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Machine-learning nowcasting of the Atlantic Meridional Overturning Circulation

Zhai,

Moradi,

Panahi

et al. 2024

APL Machine Learning

View full text Add to dashboard Cite

The Atlantic Meridional Overturning Circulation (AMOC) is a significant component of the global ocean system, which has so far ensured a relatively warm climate for the North Atlantic and mild conditions in regions, such as Western Europe. The AMOC is also critical for the global climate. The complexity of the dynamical system underlying the AMOC is so vast that a long-term assessment of the potential risk of AMOC collapse is extremely challenging. However, short-term prediction can lead to accurate estimates of the dynamical state of the AMOC and possibly to early warning signals for guiding policy making and control strategies toward preventing AMOC collapse in the long term. We develop a model-free, machine-learning framework to predict the AMOC dynamical state in the short term by employing five datasets: MOVE and RAPID (observational), AMOC fingerprint (proxy records), and AMOC simulated fingerprint and CESM AMOC (synthetic). We demonstrate the power of our framework in predicting the variability of the AMOC within the maximum prediction horizon of 12 or 24 months. A number of issues affecting the prediction performance are investigated.

show abstract

Machine-learning nowcasting of the Atlantic Meridional Overturning Circulation

Zhai,

Moradi,

Panahi

et al. 2024

APL Machine Learning

View full text Add to dashboard Cite

show abstract

Heterogeneous reinforcement learning for defending power grids against attacks

Moradi,

Panahi,

Zhai

et al. 2024

APL Machine Learning

View full text Add to dashboard Cite

Reinforcement learning (RL) has been employed to devise the best course of actions in defending the critical infrastructures, such as power networks against cyberattacks. Nonetheless, even in the case of the smallest power grids, the action space of RL experiences exponential growth, rendering efficient exploration by the RL agent practically unattainable. The current RL algorithms tailored to power grids are generally not suited when the state-action space size becomes large, despite trade-offs. We address the large action-space problem for power grid security by exploiting temporal graph convolutional neural networks (TGCNs) to develop a parallel but heterogeneous RL framework. In particular, we divide the action space into smaller subspaces, each explored by an RL agent. How to efficiently organize the spatiotemporal action sequences then becomes a great challenge. We invoke TGCN to meet this challenge by accurately predicting the performance of each individual RL agent in the event of an attack. The top performing agent is selected, resulting in the optimal sequence of actions. First, we investigate the action-space size comparison for IEEE 5-bus and 14-bus systems. Furthermore, we use IEEE 14-bus and IEEE 118-bus systems coupled with the Grid2Op platform to illustrate the performance and action division influence on training times and grid survival rates using both deep Q-learning and Soft Actor Critic trained agents and Grid2Op default greedy agents. Our TGCN framework provides a computationally reasonable approach for generating the best course of actions to defend cyber physical systems against attacks.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Machine-learning parameter tracking with partial state observation

Cited by 2 publications

References 55 publications

Machine-learning nowcasting of the Atlantic Meridional Overturning Circulation

Machine-learning nowcasting of the Atlantic Meridional Overturning Circulation

Heterogeneous reinforcement learning for defending power grids against attacks

Contact Info

Product

Resources

About