Parameter Identification of ARX Models Based on Modified Momentum Gradient Descent Algorithm

Tu, Quan; Rong, Yingjiao; Chen, Jing

doi:10.1155/2020/9537075

Cited by 16 publications

(9 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e gradient algorithm is an important tool in the linear and nonlinear problems in which a modification of parameter estimates is reached using the negative gradient direction of the criterion function. Today and in the subject of identification and control, different recursive and iterative gradient algorithms are proposed [16,17,[78][79][80][81][82][83][84]. Stochastic gradient algorithm is a basic recursive identification algorithm which is used to study different types of systems such as multivariable systems [85,86] and nonlinear block-oriented systems [16,17,26,87].…”

Section: Fsoeahw Parameter Estimationmentioning

confidence: 99%

An Improved Method for Stochastic Nonlinear System’s Identification Using Fuzzy-Type Output-Error Autoregressive Hammerstein–Wiener Model Based on Gradient Algorithm, Multi-Innovation, and Data Filtering Techniques

et al. 2021

View full text Add to dashboard Cite

This paper proposes an innovative identification approach of nonlinear stochastic systems using Hammerstein–Wiener (HW) model with output-error autoregressive (OEA) noise. Two fuzzy systems are suggested for the identification of the input and output nonlinear blocks of a proposed model from given input-output data measurements. In this work, the need for the commonly used assumptions including well-known structure of input and/or output nonlinearities and/or reversible nonlinear output is eliminated by replacing the intermediate variables and noise with their estimates. Four parametric estimation algorithms to identify the proposed fuzzy-type stochastic output-error autoregressive HW (FSOEAHW) model are derived based on backpropagation algorithm and multi-innovation and data filtering identification techniques. The proposed algorithms are improved backpropagation gradient (IBPG) algorithm, multi-innovation IBPG (MIIBPG) algorithm, a data filtering IBPG (FIBPG) algorithm, and a multi-innovation-based FIBPG (MIFIBPG) algorithm. The convergence of the parameter estimation algorithms is studied. The effectiveness of the proposed algorithms is shown by a given simulation example.

show abstract

Section: Fsoeahw Parameter Estimationmentioning

confidence: 99%

An Improved Method for Stochastic Nonlinear System’s Identification Using Fuzzy-Type Output-Error Autoregressive Hammerstein–Wiener Model Based on Gradient Algorithm, Multi-Innovation, and Data Filtering Techniques

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In this paper, the disadvantage of the SGD approach, which is the slow convergence [6,7], is noticed. To improve this weakness, the standard error from the sampling theory is added to the updating rule of the adaptive moment estimation (Adam) algorithm [8], which is the current variant of the SGD approach.…”

Section: Introductionmentioning

confidence: 98%

An Improvement of Stochastic Gradient Descent Approach for Mean-Variance Portfolio Optimization Problem

Kek

2021

Journal of Mathematics

View full text Add to dashboard Cite

In this paper, the current variant technique of the stochastic gradient descent (SGD) approach, namely, the adaptive moment estimation (Adam) approach, is improved by adding the standard error in the updating rule. The aim is to fasten the convergence rate of the Adam algorithm. This improvement is termed as Adam with standard error (AdamSE) algorithm. On the other hand, the mean-variance portfolio optimization model is formulated from the historical data of the rate of return of the S&P 500 stock, 10-year Treasury bond, and money market. The application of SGD, Adam, adaptive moment estimation with maximum (AdaMax), Nesterov-accelerated adaptive moment estimation (Nadam), AMSGrad, and AdamSE algorithms to solve the mean-variance portfolio optimization problem is further investigated. During the calculation procedure, the iterative solution converges to the optimal portfolio solution. It is noticed that the AdamSE algorithm has the smallest iteration number. The results show that the rate of convergence of the Adam algorithm is significantly enhanced by using the AdamSE algorithm. In conclusion, the efficiency of the improved Adam algorithm using the standard error has been expressed. Furthermore, the applicability of SGD, Adam, AdaMax, Nadam, AMSGrad, and AdamSE algorithms in solving the mean-variance portfolio optimization problem is validated.

show abstract

“…For example, a piecewise ARX structure was adopted to forecast the river floods; 6 a novel automated framework based on generalized spectral decomposition was proposed to estimate the parameters of an ARX model; 7 the problem of minimality and identifiability for a switching ARX system was studied; 8 the system coefficients and noise parameters of a switching ARX model were identified when the outputs are contaminated by measurement noise and process noise; 9 a modified momentum gradient descent algorithm was investigated to identify ARX models. 10 However, most of the noises considered in the aforementioned papers are white noises or Gaussian noises. Impulse noises can often be found in industrial signals, such as image signals, audio signals, and communication signals, [11][12][13] and so forth.…”

Section: Introductionmentioning

confidence: 99%

Identification of an ARX model with impulse noise using a variable step size information gradient algorithm based on the kurtosis and minimum Renyi error entropy

Jing

2021

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

ARX model is an autoregressive model with exogenous terms. Because of its simplicity and easy parameterization, the ARX model has been widely used in various applications. However, most reports on ARX identification are about Gaussian noise or white noise environment. In many practical industrial applications, impulse noise widely exists. For systems contaminated by this noise, the performance of the mean square error algorithm will deteriorate. To get more accurate results, a variable step size stochastic information gradient algorithm is proposed. The algorithm is based on the Renyi square error entropy and introduces a fourth-order statistic of the error-kurtosis-into the variable step size, which not only effectively suppresses the impulse noise, but also accelerates the convergence speed. At the same time, a simple method of determining the maximum step size is given. The computational cost and the convergence are also analyzed. Numerical experiments and case study show that for the ARX model disturbed by impulse noise, the proposed algorithm can obtain high-precision parameter estimates with fast convergence speed.

show abstract

Parameter Identification of ARX Models Based on Modified Momentum Gradient Descent Algorithm

Cited by 16 publications

References 55 publications

An Improved Method for Stochastic Nonlinear System’s Identification Using Fuzzy-Type Output-Error Autoregressive Hammerstein–Wiener Model Based on Gradient Algorithm, Multi-Innovation, and Data Filtering Techniques

An Improved Method for Stochastic Nonlinear System’s Identification Using Fuzzy-Type Output-Error Autoregressive Hammerstein–Wiener Model Based on Gradient Algorithm, Multi-Innovation, and Data Filtering Techniques

An Improvement of Stochastic Gradient Descent Approach for Mean-Variance Portfolio Optimization Problem

Identification of an ARX model with impulse noise using a variable step size information gradient algorithm based on the kurtosis and minimum Renyi error entropy

Contact Info

Product

Resources

About