Development of an operation trajectory design algorithm for control of multiple 0D parameters using deep reinforcement learning in KSTAR

Seo, Jaemin; Na, Yong-Su; Kim, Boseong; Lee, Chan-Young; Park, Minseo; Park, Sangjin; Lee, Yeong Ho

doi:10.1088/1741-4326/ac79be

Cited by 20 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, various machine learning (ML) approaches have also been explored in magnetic confinement fusion research for magnetic field reconstruction or prediction, and they have been applied in various scenarios, including equilibrium reconstruction and solver [15][16][17][18], plasma control [19][20][21][22][23][24][25][26], etc. From neural networks [27] to reinforcement learning [4,28,29], various data-driven ML methods have achieved promising results in the field of magnetic confinement. However, few studies [4,30] focused on the LCFS modeling/prediction.…”

Section: Introductionmentioning

confidence: 99%

Predict the last closed-flux surface evolution without physical simulation

Wan,

Bai,

et al. 2024

Nucl. Fusion

View full text Add to dashboard Cite

One of the main challenges in developing effective control strategies for the magnetic control system in tokamaks has been the difficulty in obtaining the last closed-flux surface (LCFS) evolution results from control commands. We have developed a data-driven model that combines a predictive model and a surrogate model for physics simulation programs. This model is capable of predicting the LCFS without relying on physical simulation codes. Addressing the data characteristics of LCFS, we have proposed a specialized discretization approach to achieve dimensionality reduction. Furthermore, we have excluding the control references, the model can be seamlessly integrated into the control system, providing real-time LCFS prediction. Following comprehensive testing and multifaceted evaluation, our model has demonstrated highly satisfactory results of 95% or above, meeting practical requirements.

show abstract

Section: Introductionmentioning

confidence: 99%

Predict the last closed-flux surface evolution without physical simulation

Wan,

Bai,

et al. 2024

Nucl. Fusion

View full text Add to dashboard Cite

show abstract

“…Moreover, it has exhibited significant advantages in avoidance control problems [25], which is essentially similar to the objective of this work. Recently, RL has been applied to tokamak control and optimization, demonstrating promising achievements [26][27][28][29][30][31]. The RL algorithm optimizes the actor model based on a deep neural network (DNN), and the actor model gradually learns the action policy leading to higher rewards in a given environment.…”

Section: Introductionmentioning

confidence: 99%

Avoiding tokamak tearing instability with artificial intelligence

Kolemen

Seo

Conlin

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

For stable and efficient fusion energy production using a tokamak reactor, maintaining high-pressure hydrogenic plasma without plasma disruption is essential. Therefore, it is necessary to actively control the tokamak based on the observed plasma state, to maneuver high-pressure plasma while avoiding tearing instability, the leading cause of disruptions. This presents an obstacle avoidance problem for which artificial intelligence (AI) based on reinforcement learning has recently shown remarkable performance. However, the obstacles here, the tearing instability, are difficult to forecast and highly prone to terminating plasma operations. In our recent work, we developed a multimodal dynamic model that estimates the likelihood of future tearing instability based on signals from multiple diagnostics and actuators. This dynamic model not only predicts the possible onset of tearing instability during tokamak operation but can also be used as a training environment for AI that controls actuators to avoid instabilities. In this work, we demonstrate AI control based on reinforcement learning to lower the possibility of disruptive tearing instabilities in DIII-D, the largest magnetic fusion facility in the US. The controller maintained the tearing likelihood under a given threshold, under relatively unfavorable conditions of low safety factor and low torque.

show abstract

“…Furthermore, it demonstrated the concept of fully automated reconstruction that can be applied to plasma prediction and control. Recently, studies on real-time plasma profile prediction [13], control [14,15] and profile-based instability avoidance [16] have been initiated. Real-time capable kinetic equilibrium reconstruction technology will further advance plasma profile-based prediction and control.…”

Section: Introductionmentioning

confidence: 99%

Machine learning-based real-time kinetic profile reconstruction in DIII-D

Shousha,

Seo,

Erickson

et al. 2023

Nucl. Fusion

Self Cite

View full text Add to dashboard Cite

Kinetic equilibrium reconstruction plays a vital role in the physical analysis of plasma stability and control in fusion tokamaks. However, the traditional approach is subjective and prone to human biases. To address this, the consistent automatic kinetic equilibrium reconstruction (CAKE) method was introduced, providing objective results. Nonetheless, its offline nature limits its application in real-time plasma control systems (PCSs). To address this limitation, we present RTCAKENN, a machine learning model that approximates 7 CAKE-level output profiles, namely pressure, inverse q, toroidal current density, electron temperature and density, carbon ion impurity temperature and rotation profiles, using real-time available inputs. The deep neural network consists of an encoder layer, where the scalars and interdependent inputs such as plasma boundary coordinates and motional Stark effect data are encoded using multi-layer perceptrons (MLPs), while profile inputs are encoded by 1D convolutional layers. The encoded data is passed through a MLP for latent feature extraction, before being decoded in the decoding layers, which consist of upsampling and convolutional layers. RTCAKENN has been implemented in the DIII-D PCS and our model achieves accuracy comparable to CAKE and surpasses existing real-time alternatives. Through clever dropout training, RTCAKENN exhibits robustness and can operate even in the absence of Thomson scattering data or charge exchange recombination data. It executes in under 8 ms in the real-time environment, enabling future application in real-time control and analysis.

show abstract

Development of an operation trajectory design algorithm for control of multiple 0D parameters using deep reinforcement learning in KSTAR

Cited by 20 publications

References 18 publications

Predict the last closed-flux surface evolution without physical simulation

Predict the last closed-flux surface evolution without physical simulation

Avoiding tokamak tearing instability with artificial intelligence

Machine learning-based real-time kinetic profile reconstruction in DIII-D

Contact Info

Product

Resources

About