Analysis of a nonsmooth optimization approach to robust estimation

Bako, Laurent

doi:10.1016/j.automatica.2015.12.024

Cited by 15 publications

(29 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the help of the device of self-decomposability amplitude (12), we can state a condition for exact recovery of the parameter matrix A o by solving the optimization problem in (3). A similar result was proven in [3] for the Least Absolute Deviation (LAD) estimator.…”

Section: A Exact Recoverabilitysupporting

confidence: 73%

On a Class of Optimization-Based Robust Estimators

Bako

2017

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

We consider in this paper the problem of estimating a parameter matrix from observations which are affected by two types of noise components: (i) a sparse noise sequence which, whenever nonzero can have arbitrarily large amplitude (ii) and a dense and bounded noise sequence of "moderate" amount. This is termed a robust regression problem. To tackle it, a quite general optimization-based framework is proposed and analyzed. When only the sparse noise is present, a sufficient bound is derived on the number of nonzero elements in the sparse noise sequence that can be accommodated by the estimator while still returning the true parameter matrix. While almost all the restricted isometrybased bounds from the literature are not verifiable, our bound can be easily computed through solving a convex optimization problem. Moreover, empirical evidence tends to suggest that it is generally tight. If in addition to the sparse noise sequence, the training data are affected by a bounded dense noise, we derive an upper bound on the estimation error.

show abstract

Section: A Exact Recoverabilitysupporting

confidence: 73%

On a Class of Optimization-Based Robust Estimators

Bako

2017

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Note that these problems are also of interest for the robust estimation of a single (non-hybrid) linear model in the presence of outliers, as will be considered in the examples of Sect. 5.3-5.4 (see also [4] for an analysis of the sparse optimization method of [3] in this context).…”

Section: Bounded-error Estimationmentioning

confidence: 99%

“…In this case, the problems (39) or (40) are solved only once to estimate a single model from the maximal number of points that can be considered as inliers. We compare the proposed algorithms with the standard 1 -minimization in a setting similar to the one in [4]: an increasing fraction r of 500 data points in dimension 4 are corrupted by outliers ζ i drawn from a Gaussian distribution with mean 100 and standard deviation 1000: y i = θ T x i + ξ i + ζ i . The results are reported in Fig.…”

Section: Bounded-error Estimation In the Presence Of Outliersmentioning

confidence: 99%

“…Most of the analysis of convex relaxations in [4] actually deals with exact recovery in the case where the data is noiseless (ξ i = 0) except for sparse and gross errors. The proposed boundederror algorithms can also be applied in this setting, e.g., with = 10 −6 , and evaluated on the basis Linear case 98% 98% 80% Affine case with positive outliers 98% 98% 43% of the probabililty of exact recovery, estimated as the fraction of successful trials 4 .…”

Section: Exact Recovery With Sparse and Gross Measurement Errorsmentioning

confidence: 99%

See 1 more Smart Citation

Global optimization for low-dimensional switching linear regression and bounded-error estimation

Lauer

2018

Automatica

View full text Add to dashboard Cite

The paper provides global optimization algorithms for two particularly difficult nonconvex problems raised by hybrid system identification: switching linear regression and bounded-error estimation. While most works focus on local optimization heuristics without global optimality guarantees or with guarantees valid only under restrictive conditions, the proposed approach always yields a solution with a certificate of global optimality. This approach relies on a branchand-bound strategy for which we devise lower bounds that can be efficiently computed. In order to obtain scalable algorithms with respect to the number of data, we directly optimize the model parameters in a continuous optimization setting without involving integer variables. Numerical experiments show that the proposed algorithms offer a higher accuracy than convex relaxations with a reasonable computational burden for hybrid system identification. In addition, we discuss how bounded-error estimation is related to robust estimation in the presence of outliers and exact recovery under sparse noise, for which we also obtain promising numerical results.arXiv:1707.05533v3 [cs.LG] 23 Nov 2017and the algorithm stops at iteration T . ConvergenceAs for the switching regression case studied in Sect. 3.3, convergence is obtained, for both the 2, and the 0, loss functions, from the tightness of the bounds, under the following assumptions.Assumption 3. For p = 2 (respectively, p = 0), the global optimum of Problem (39) (resp. Problem (40)) is strictly positive:Assumption 4. For any p ∈ {0, 2}, upper bounds in a box B = [u, v] are computed as J(B) = J p BE (u) or such that J(B) ≤ J p BE (u). Theorem 2. Under Assumptions 3-4, the branch-and-bound algorithm described above to minimize J p BE with p ∈ {0, 2} and lower bounds J(B) computed as in Lemma 6 for p = 2 or Lemma 7 for p = 0 converges in a finite number of iterations for any T OL > 0.

show abstract

“…In this context, relying on suboptimal solutions can lead to highly unsatisfactory results with many misclassifications of data points. Robust methods based on convex relaxations (Liu et al, 2013;Bako, 2014;Bako and Ohlsson, 2016) or iteratively hard-thresholding (Bhatia et al, 2015) offer some guarantees but are only optimal under particular conditions on the data. Instead, in this paper, we aim at unconditional optimality and discuss the computational complexity of globally minimizing a saturated loss function for the robust estimation of linear models, let it be regression ones or subspaces.…”

Section: Introductionmentioning

confidence: 99%

On the exact minimization of saturated loss functions for robust regression and subspace estimation

Lauer

2018

Pattern Recognition Letters

View full text Add to dashboard Cite

This paper deals with robust regression and subspace estimation and more precisely with the problem of minimizing a saturated loss function. In particular, we focus on computational complexity issues and show that an exact algorithm with polynomial time-complexity with respect to the number of data can be devised for robust regression and subspace estimation. This result is obtained by adopting a classification point of view and relating the problems to the search for a linear model that can approximate the maximal number of points with a given error. Approximate variants of the algorithms based on ramdom sampling are also discussed and experiments show that it offers an accuracy gain over the traditional RANSAC for a similar algorithmic simplicity.

show abstract

Analysis of a nonsmooth optimization approach to robust estimation

Cited by 15 publications

References 42 publications

On a Class of Optimization-Based Robust Estimators

On a Class of Optimization-Based Robust Estimators

Global optimization for low-dimensional switching linear regression and bounded-error estimation

On the exact minimization of saturated loss functions for robust regression and subspace estimation

Contact Info

Product

Resources

About