Study on Prediction of Zinc Grade by Transformer Model with De-Stationary Mechanism

Abstract

At present, in the mineral flotation process, flotation data are easily influenced by various factors, resulting in non-stationary time series data, which lead to overfitting of prediction models, ultimately severely affecting the accuracy of grade prediction. Thus, this study proposes a de-stationary attention mechanism based on the transformer model (DST) to learn non-stationary information in raw mineral data sequences. First, normalization processing is performed on matched flotation data and mineral grade values, to make the data sequences stationary, thereby enhancing model prediction capabilities. Then, the proposed de-stationary attention mechanism is employed to learn the temporal dependencies of mineral flotation data in the transformed vanilla transformer model, i.e., non-stationary information in the mineral data sequences. Lastly, de-normalization processing is conducted to maintain the mineral prediction results within the same scale as the original data. Compared with existing models such as RNN, LSTM, transformer, Enc-Dec (RNN), and STS-D, the DST model reduced the RMSE by 20.8%, 20.8%, 62.8%, 20.5%, and 49.1%, respectively.