Multivariable System Prediction Based on TCN-LSTM Networks with Self-Attention Mechanism and LASSO Variable Selection

Abstract

Intelligent prediction of key output variables that are difficult to measure online in complex systems has important research significance. In this paper, by using the least absolute shrinkage and selection operator (LASSO) algorithm to analyze the principal elements of input variables, a temporal convolutional network fused with long short-term memory (TCN-LSTM) network and self-attention mechanism (SAM) is designed to realize dynamic modeling of multivariate feature sequences. For complex processes with multiple input variables, each variable has different effects on the output, so it is necessary to use the LASSO algorithm to perform regression analysis on the input and output data for selecting the principal component variables and reducing the redundancy and computation burden of the network. The TCN network is used to extract the features of the input variables efficiently. The long-term memory performance of time series is enhanced by applying an LSTM network. The multihead SAM is used to optimize the network, and the role of key features is enhanced by assigning weights with probability to further improve the accuracy of sequence prediction. Finally, by comparison with the existing network model, the offline data generated by the high and low converters in the synthetic ammonia industry is used to predict the CO content so as to verify the superiority and applicability of the proposed network model.

Simultaneous Determination of Chemical Oxygen Demand, Total Nitrogen, Ammonia, and Phosphate in Surface Water Based on a Multielectrode System

“…To address the issue of overfitting in multivariable models, penalty regression estimation methods such as LASSO have gained significant popularity as a feature variable extraction technique . To implement the LASSO regression, an L1 regularization term is incorporated into the objective function.…”

“…To address the issue of overfitting in multivariable models, penalty regression estimation methods such as LASSO have gained significant popularity as a feature variable extraction technique. 44 To implement the LASSO regression, an L1 regularization term is incorporated into the objective function. The L1 term represents the absolute value of the regression coefficient multiplied by the sum of regularization parameters, thereby controlling the strength of regularization.…”