On a Class of Optimization-Based Robust Estimators

Abstract

We consider in this paper the problem of estimating a parameter matrix from observations which are affected by two types of noise components: (i) a sparse noise sequence which, whenever nonzero can have arbitrarily large amplitude (ii) and a dense and bounded noise sequence of "moderate" amount. This is termed a robust regression problem. To tackle it, a quite general optimization-based framework is proposed and analyzed. When only the sparse noise is present, a sufficient bound is derived on the number of nonzero elements in the sparse noise sequence that can be accommodated by the estimator while still returning the true parameter matrix. While almost all the restricted isometrybased bounds from the literature are not verifiable, our bound can be easily computed through solving a convex optimization problem. Moreover, empirical evidence tends to suggest that it is generally tight. If in addition to the sparse noise sequence, the training data are affected by a bounded dense noise, we derive an upper bound on the estimation error.

An optimization framework for resilient batch estimation in Cyber-Physical Systems

“…Hence the estimation of the state trajectory reduces to estimating the initial state x 0 . This can be viewed as a robust regression problem, like the ones discussed in [12], [2]. Generalizing a result in [2], we derive next a necessary and sufficient condition for exact recovery of the true state, which holds if and only if arg min z∈R n V • Σ (Y, z) = {x 0 } with x 0 being the exact initial state of the system Σ.…”

“…This can be viewed as a robust regression problem, like the ones discussed in [12], [2]. Generalizing a result in [2], we derive next a necessary and sufficient condition for exact recovery of the true state, which holds if and only if arg min z∈R n V • Σ (Y, z) = {x 0 } with x 0 being the exact initial state of the system Σ. To this end, we first introduce the concept of concentration ratio of a collection of matrices with respect to a loss function.…”

Robustness analysis of a maximum correntropy framework for linear regression

“…Theorem 8 Let I 0 ε = {t ∈ I : |v t | ≤ ε} and I c ε = {t ∈ I : |v t | > ε} with {v t } denoting the noise sequence in (1). Let ℓ be a function obeying P1-P4.…”

Analysis of resilience for a State Estimator for Linear Systems

“…Here, λ > 0 is a user-defined parameter which aims at balancing the contributions of the two terms involved in the expression of the performance index F . This idea of weighting the terms contained in F could also be done differently depending on the time index, for example by taking terms of the form W t (z t+1 − Az t ) 2 2 and V t (y t − Cz t ) 1 , where W t and V t would be positive-definite weighting matrices.…”

“…This is indeed a common characteristic of the concepts which are usually used to assess resilience; for example the popular Restricted Isometry Property (RIP) constant [3] is comparatively as hard to evaluate. Nevertheless, if we restrict our attention to estimation problems where the process noise {w t } would be identically equal to zero, then by adding in (9) the additional constraint that z t+1 = Az t , p r can be exactly computed using the method in [20] or more cheaply overestimated using the one in [2].…”