Hybrid Series/Parallel All-Nonlinear Dynamic-Static Neural Networks: Development, Training, and Application to Chemical Processes

Abstract

This paper presents the development of data-driven hybrid nonlinear static-nonlinear dynamic neural network models and addresses the challenges of optimal estimation of parameters for such hybrid networks. A parallel static-dynamic neural network and two variants of series networks, specifically, nonlinear static-nonlinear dynamic and nonlinear dynamic-nonlinear static networks, are investigated in this work. Performances of the proposed fully nonlinear hybrid series and parallel network models are compared with the existing state-of-the-art data-driven models like long−short-term memory networks and gated recurrent unit models as well as DABNet-type linear-dynamic-nonlinear-static neural network. Algorithms are developed for training the series and parallel hybrid networks where the static and dynamic networks can be trained independently by different algorithms. These algorithms offer flexibility for incorporating different types of static and dynamic network architectures and their training algorithms thus offering tradeoff between computational expense and accuracy for highly nonlinear systems. In addition to the typical training objective that minimizes some squared error, an objective function is considered that penalizes overfitting and uncertainty in parameter estimates. Performances of the proposed algorithms are evaluated for three nonlinear example problems�a two-tank pH neutralization reactor, the widely used Van de Vusse reactor, and a pilot plant for postcombustion CO 2 capture using monoethanolamine solvent. Computational expense and convergence performance of the proposed network architectures and their training algorithms are presented.

Hot Corrosion Monitoring in Boilers Using a Nonlinear Estimator and Electrochemical Noise-Based Corrosion Sensors

“…Data-driven models are developed for O 2 conservation and metal temperature using feedforward neural networks. , The three-layered feedforward neural network chosen comprises an input layer, a single hidden layer, and an output layer, whose architecture is presented in Figure .…”

Liquid–Liquid Dispersion Performance Prediction and Uncertainty Quantification Using Recurrent Neural Networks

“…16 Other RNN variations have been tailored for specific applications, thus making them less applicable for chemical engineering processes, such as Hopfield networks and NARX-type RNNs for identification and control of nonlinear dynamic systems, and adaptive networks like ABAM and transversal/recursive filters for signal processing and communications. 17 Diving into one of the most popular variants of RNNs, LSTM networks, designed to solve the so-called vanishing problem 18,19 seen in the "vanilla" RNNs, have been commonly used in a wide range of applications. For instance, LSTM networks have been trained using simulation data to predict scenarios including material leakage position 20,21 and bio-oil yield of a fluidized bed.…”

Hybrid Modeling of Methanol to Olefin Reaction Kinetics Based on the Artificial Neural Network

“…Hybrid modeling approaches developed in recent years expect to address the issues of accuracy, real-time, and simplicity. − The hybrid modeling is a combination of the first-principles model (FPM) and the data-driven model (DDM). In hybrid modeling, the DDM based on the machine learning approach can replace complex and unimportant mechanistic parts of the FPM.…”