A new method for estimation and model selection: $$\rho $$ ρ -estimation

Abstract

Abstract. The aim of this paper is to present a new estimation procedure that can be applied in various statistical frameworks including density and regression and which leads to both robust and optimal (or nearly optimal) estimators. In density estimation, they asymptotically coincide with the celebrated maximum likelihood estimators at least when the statistical model is regular enough and contains the true density to estimate. For very general models of densities, including non-compact ones, these estimators are robust with respect to the Hellinger distance and converge at optimal rate (up to a possible logarithmic factor) in all cases we know. In the regression setting, our approach improves upon the classical least squares in many respects. In simple linear regression for example, it provides an estimation of the coefficients that are both robust to outliers and simultaneously rateoptimal (or nearly rate-optimal) for a large class of error distributions including Gaussian, Laplace, Cauchy and uniform among others.

Mean Estimation and Regression Under Heavy-Tailed Distributions: A Survey

“…Note that Theorem 12 shows that Ψ n is a desired norm oracle: if Ψ n (h) > c 6 Bρ then it follows that h L 2 ≥ ρ, and thus…”

Learning from MOM’s principles: Le Cam’s approach

“…Baraud and Birgé (2009) ;Birgé (2006Birgé ( , 2013) and by Baraud, Birgé and Sart to define ρ-estimators (cf. Baraud and Birgé (2016); Baraud et al (2017)). Baraud (2011); Baraud et al (2014) also built efficient estimator selection procedures with this approach.…”

About the non-asymptotic behaviour of Bayes estimators

“…Our next result is a suitable version, due to Yannick Baraud, of a lemma of Barron (1991, Section 5.3) which appears as Proposition 1 in Baraud, Birgé and Sart (2014).…”

“…But, unlike for universally robust estimators like the T-estimators of Birgé (2006) and the ρ-estimators of Baraud, Birgé and Sart (2014), we have to measure the distorsion between S and P not in terms of the Hellinger distance, as would be the case for the previous estimators, but in terms of the Kullback-Leibler divergence, as is the case for the maximum likelihood estimator -see for instance Massart (2007) -. Assumptions involving the KullbackLeibler divergence also appear in other works on Bayesian estimators like Ghosal, Gosh and van der Vaart (2000).…”