Fully Convolutional Spatio-Temporal Models for Representation Learning in Plasma Science

Abstract

We have trained a fully convolutional spatio-temporal model for fast and accurate representation learning in the challenging exemplar application area of fusion energy plasma science. The onset of major disruptions is a critically important fusion energy science issue that must be resolved for advanced tokamak plasmas such as the $25B burning plasma international thermonuclear experimental reactor (ITER) experiment. While a variety of statistical methods have been used to address the problem of tokamak disruption prediction and control, recent approaches based on deep learning have proven particularly compelling. In the present paper, we introduce further improvements to the fusion recurrent neural network (FRNN) software suite, which delivered cross-machine disruption predictions with unprecedented accuracy using a large database of experimental signals from two major tokamaks. Up to now, FRNN was based on the long short-term memory (LSTM) variant of recurrent neural networks to leverage the temporal information in the data. Here, we implement and apply the "temporal convolutional neural network (TCN)" architecture to the time-dependent input signals. This allows highly optimized convolution operations to carry the majority of the computational load of training, thus enabling a reduction in training time, and the effective use of highperformance computing resources for hyperparameter tuning. At the same time, the TCN-based architecture achieves better predictive performance when compared with the LSTM architecture for various tasks for a representative fusion database.

Implementation of AI/DEEP learning disruption predictor into a plasma control system

“…As explained there, all inputs are normalized to the same order of magnitude, the variance of the input quantities is appropriately scaled, and the values submitted during real‐time are correlated with those trained offline. In particular, the 1D inputs are processed by a set of convolutional neural nets and then concatenated with the 0D inputs to form the input features for the long short‐term memory (LSTM) network as well as for the temporal convolutional neural (TCN) network discussed in Reference [10] The outputs of the LSTM and the TCN in these studies provide complementary benchmarks for the disruption score indicating proximity of the coming disruption event.…”

“…[6,7] It is particularly noteworthy that in addressing this long-standing challenge, neural networks, which were considered for decades, [8] have recently taken a dramatic step forward with the advent of more powerful artificial intelligence approaches enabled by the rapid advances in high-performance computing technology at major supercomputing centers. For example, deep learning models based on the long-short term memory (LSTM) recurrent neural network (RNN) [9] and temporal convolutional neural networks (TCN) [10] have achieved breakthrough results for cross-machine predictions with the aid of leadership-class high-performance-computing (HPC) facilities. [11] The state-of-the-art deep learning disruption prediction models based on the Fusion Recurrent Neural Network (FRNN) [9] have been further improved.…”

An Oracle for Guiding Large-Scale Model/Hybrid Parallel Training of Convolutional Neural Networks

“…Inference at large scale is becoming increasingly demanded, given that for large models the inference would also be distributed [4]. In smaller models, when latency of inference matters in an application, the inference could also be distributed (e.g., real-time prediction of Tokamak disruptions in magnetically-confined thermonuclear plasma experiments [11]). Some of the limitations and bottlenecks of distributed training discussed previously also appear in distributed inference (See lines marked with Y in column I of Table 6).…”

Exploration of Quantum Machine Learning and AI Accelerators for Fusion Science

“…The goal of such predictions is to enable interventions that mitigate or avoid the detrimental effects of such disruptions on the fusion reactors. The recent state-of-the-art architecture utilizes scalar inputs, including plasma current and internal inductance, as well as 1D electron temperature and density profiles [12,8]. For each time step, the 1D profiles are processed using multichannel classical convolution, followed by concatenation with the scalar signals.…”

“…The earliest work uses an long short-term memory (LSTM) architecture to propagate information from the past in order to inform the prediction [12]. A more recent iteration uses temporal convolutional networks (TCN) to again a much larger memory capacity compared to recurrent models [8]. The above summary points out two ways we can insert quantum convolution: replacing spatial convolution on the 1D profiles with quantum convolution and replacing temporal convolution with quantum temporal convolution.…”