Minimum-entropy estimation in semi-parametric models

Wolsztynski, Eric; Thierry, Éric; Pronzato, Luc

doi:10.1016/j.sigpro.2004.11.028

Cited by 42 publications

(27 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, Wolsztynski et al (2005) proposed a minimum-entropy estimation of the regression problem considered here. They chooseθ n to minimizeĤ n (θ)=− R An…”

Section: Some Related Methodsmentioning

confidence: 99%

Semiparametric Regression with Kernel Error Model

Yuan

Gooijer

2006

SSRN Journal

View full text Add to dashboard Cite

Standard-Nutzungsbedingungen:Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden.Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen.Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. ABSTRACT. We propose and study a class of regression models, in which the mean function is specified parametrically as in the existing regression methods, but the residual distribution is modeled nonparametrically by a kernel estimator, without imposing any assumption on its distribution. This specification is different from the existing semiparametric regression models. The asymptotic properties of such likelihood and the maximum likelihood estimate (MLE) under this semiparametric model are studied. We show that under some regularity conditions, the MLE under this model is consistent (as compared to the possibly pseudo consistency of the parameter estimation under the existing parametric regression model), and is asymptotically normal with rate √ n and efficient. The nonparametric pseudo-likelihood ratio has the Wilks property as the true likelihood ratio does. Simulated examples are presented to evaluate the accuracy of the proposed semiparametric MLE method. Terms of use: Documents in

show abstract

“…More recently, Wolsztynski et al (2005) proposed a minimum-entropy estimation of the regression problem considered here. They chooseθ n to minimizeĤ n (θ)=− R An…”

Section: Some Related Methodsmentioning

confidence: 99%

Semiparametric Regression with Kernel Error Model

Yuan

Gooijer

2006

SSRN Journal

View full text Add to dashboard Cite

show abstract

“…The robust experimental design methods have been applied to problems containing both static and dynamic systems identification [9]. For static systems, for which the response surface is known up to vector of parameters and the errors have an unknown density, the minimum entropy parametric estimator should be considered [10,11]. The idea of such an approach is to minimise the estimated entropy of the distribution of the residuals.…”

Section: Open Accessmentioning

confidence: 99%

Plant Friendly Input Design for Parameter Estimation in an Inertial System with Respect to D-Efficiency Constraints

Jakowluk

2014

Entropy

View full text Add to dashboard Cite

System identification, in practice, is carried out by perturbing processes or plants under operation. That is why in many industrial applications a plant-friendly input signal would be preferred for system identification. The goal of the study is to design the optimal input signal which is then employed in the identification experiment and to examine the relationships between the index of friendliness of this input signal and the accuracy of parameter estimation when the measured output signal is significantly affected by noise. In this case, the objective function was formulated through maximisation of the Fisher information matrix determinant (D-optimality) expressed in conventional Bolza form. As setting such conditions of the identification experiment we can only talk about the D-suboptimality, we quantify the plant trajectories using the D-efficiency measure. An additional constraint, imposed on D-efficiency of the solution, should allow one to attain the most adequate information content from the plant which operating point is perturbed in the least invasive (most friendly) way. A simple numerical example, which clearly demonstrates the idea presented in the paper, is included and discussed.

show abstract

“…The minimum error entropy (MEE) [18][19][20][21][22][23][24][25][26][27] criterion in ITL was successfully used in adaptive filtering to improve the learning performance in non-Gaussian noises. Basically, the MEE aims at minimizing the entropy of the training error such that the adaptive model becomes as close as possible to the unknown system.…”

Section: Introductionmentioning

confidence: 99%

Minimum Error Entropy Algorithms with Sparsity Penalty Constraints

Peng

et al. 2015

Entropy

View full text Add to dashboard Cite

Abstract:Recently, sparse adaptive learning algorithms have been developed to exploit system sparsity as well as to mitigate various noise disturbances in many applications. In particular, in sparse channel estimation, the parameter vector with sparsity characteristic can be well estimated from noisy measurements through a sparse adaptive filter. In previous studies, most works use the mean square error (MSE) based cost to develop sparse filters, which is rational under the assumption of Gaussian distributions. However, Gaussian assumption does not always hold in real-world environments. To address this issue, we incorporate in this work an l1-norm or a reweighted l1-norm into the minimum error entropy (MEE) criterion to develop new sparse adaptive filters, which may perform much better than the MSE based methods, especially in heavy-tailed non-Gaussian situations, since the error entropy can capture higher-order statistics of the errors. In addition, a new approximator of l0-norm, based on the correntropy induced metric (CIM), is also used as a sparsity penalty term (SPT). We analyze the mean square convergence of the proposed new sparse adaptive filters. An energy conservation relation is derived and a sufficient condition is obtained, which ensures the mean square convergence. Simulation results confirm the superior performance of the new algorithms. OPEN ACCESSEntropy 2015, 17 3420 Keywords: sparse estimation; minimum error entropy; correntropy induced metric; mean square convergence; impulsive noise MSC Codes: 62B10

show abstract

Minimum-entropy estimation in semi-parametric models

Cited by 42 publications

References 19 publications

Semiparametric Regression with Kernel Error Model

Semiparametric Regression with Kernel Error Model

Plant Friendly Input Design for Parameter Estimation in an Inertial System with Respect to D-Efficiency Constraints

Minimum Error Entropy Algorithms with Sparsity Penalty Constraints

Contact Info

Product

Resources

About