A Novel Recurrent Neural Network-Based Ultra-Fast, Robust, and Scalable Solver for Inverting a “Time-Varying Matrix”

Abstract

The concept presented in this paper is based on previous dynamical methods to realize a time-varying matrix inversion. It is essentially a set of coupled ordinary differential equations (ODEs) which does indeed constitute a recurrent neural network (RNN) model. The coupled ODEs constitute a universal modeling framework for realizing a matrix inversion provided the matrix is invertible. The proposed model does converge to the inverted matrix if the matrix is invertible, otherwise it converges to an approximated inverse. Although various methods exist to solve a matrix inversion in various areas of science and engineering, most of them do assume that either the time-varying matrix inversion is free of noise or they involve a denoising module before starting the matrix inversion computation. However, in the practice, the noise presence issue is a very serious problem. Also, the denoising process is computationally expensive and can lead to a violation of the real-time property of the system. Hence, the search for a new ‘matrix inversion’ solving method inherently integrating noise-cancelling is highly demanded. In this paper, a new combined/extended method for time-varying matrix inversion is proposed and investigated. The proposed method is extending both the gradient neural network (GNN) and the Zhang neural network (ZNN) concepts. Our new model has proven that it has exponential stability according to Lyapunov theory. Furthermore, when compared to the other previous related methods (namely GNN, ZNN, Chen neural network, and integration-enhanced Zhang neural network or IEZNN) it has a much better theoretical convergence speed. To finish, all named models (the new one versus the old ones) are compared through practical examples and both their respective convergence and error rates are measured. It is shown/observed that the novel/proposed method has a better practical convergence rate when compared to the other models. Regarding the amount of noise, it is proven that there is a very good approximation of the matrix inverse even in the presence of noise.

Facilitating Inversion of the Error Covariance Models for the Wide-Swath Altimeters

“…In a more general perspective, we assume that obtaining a universal and numerically efficient algorithm for rescaling the wide-swath altimetry observations is a challenging problem that could be solved by jointly analyzing real data from the forthcoming missions and ensemble simulations of the ocean surface topography by state-of-the-art numerical models. Highly structured observation error covariances of the wide-swath altimetry provide an opportunity to apply deep learning techniques (e.g., [15,16]) for retrieval of both more realistic estimates of the covariance matrices and their inverses. Extensive literature (e.g., [17][18][19][20]) also exists on other methods of approximating the inverse covariances.…”

“…The system of Equations (18) and (19) is non-linear and has multiple solutions (S = α q = 0 is one of those). To avoid this ambiguity, a good first guess for the control vector {S, α q } can be prepared by first rescaling the spectral components S q to minimize the distance between them by choosing a suboptimal α q , and then averaging the resulting scaled approximations of S q in order to get a plausible first guess for S. In the simple case when the along-track spectra can be rescaled with zero error (S q = α q S), Equations (16) and (17) are reduced to identities.…”

“…Since the exact expressions for the cost function (15) and gradients (16) and (17) are available, minimization of (15) can be done at relatively low expense by a quasi-Newtonian algorithm. The above expressions depend only on the four structure functions f q and four along-track spectra S q , that are easy to manipulate.…”

Order reduction of nonlinear quasi-periodic systems subjected to external excitations

“…Neural Network: One can also use a dynamical method using a recurrent neural network proposed for inversion of the time-varying matrix. One could use the gradient method [34], Zhang dynamics [35][36][37], or Chen dynamics [38] to find an inverse. In this section, we briefly present the Zhang dynamics approach [39] that could be used for inverting the L-P transformation.…”

Intelligent Transportation Related Complex Systems and Sensors

“…Building upon dynamical methods to realize a time-varying matrix inversion, [ 24 ] relies on a system of coupled ordinary differential equations (ODEs), constituting a recurrent neural network (RNN) model as a universal modelling framework for realizing a matrix inversion, provided the matrix is invertible. The study in this paper builds around this framework to propose and investigate a new combined/extended method for time-varying matrix inversion that extends both the gradient neural network (GNN) and the Zhang neural network (ZNN) concepts.…”