Robust RLS via the nonconvex sparsity prompting penalties of outlier components

Robust Hierarchical-Optimization RLS Against Sparse Outliers

“…To address this "curse of dimensionality," studies [12,13] adopt the following recursions for all n ≥ n 0 + 1,ô…”

“…Methods that strengthen RLS against outliers have been reported in [7][8][9][10][11][12][13]. Propelled by robustregression arguments [3], studies [7][8][9] utilize M-estimate losses instead of typical LS ones to penalize system-output errors.…”

“…Outliers are modeled as nuisance variables and are jointly estimated with the filter's coefficients in [12,13]. To address identifiability issues in the estimation task due to the ever-growing number of the outlier unknowns with recursions (time instances), filter and outlier vectors are updated per recursion via the minimization of an LS data-fit loss plus a surrogate of the 0 -norm of the outlier vector to exploit sparsity: The 1 -norm is utilized in [12], while non-convex surrogates in [13]. The composite minimization task is solved in an alternating fashion: First, the outlier vector is updated, and then the classical RLS is used to update the filter's estimates.…”

“…This short paper follows the path of [12,13] to model outliers as nuisance variables, but employs the recently introduced hierarchical(-optimization) recursive least squares (HO-RLS) [16] to update filter coefficients instead of the classical RLS. Unlike RLS, which propels all of [7][8][9][10][11][12][13], HO-RLS provides a way to quantify side information about the system since it solves a hierarchical-optimization problem: Minimize a convex loss, which models the available side information, over the minimizers of the classical ensemble LS data-fit loss.…”

“…This short paper follows the path of [12,13] to model outliers as nuisance variables, but employs the recently introduced hierarchical(-optimization) recursive least squares (HO-RLS) [16] to update filter coefficients instead of the classical RLS. Unlike RLS, which propels all of [7][8][9][10][11][12][13], HO-RLS provides a way to quantify side information about the system since it solves a hierarchical-optimization problem: Minimize a convex loss, which models the available side information, over the minimizers of the classical ensemble LS data-fit loss. The proposed outlier-robust HO-RLS builds on steepestdescent directions with a constant step size (learning rate), needs no matrix inversion (lemma), exhibits similar computational complexity with the implementations in [12,13], accommodates colored noise of known covariance matrix without any prewhitening, and offers theoretical guarantees, in a probabilistic sense, for the convergence of the filter/system estimates to the solution of the aforementioned hierarchical-optimization task.…”