Outlier-Robust Kernel Hierarchical-Optimization RLS on a Budget with Affine Constraints

Abstract

This paper introduces a non-parametric learning framework to combat outliers in online, multi-output, and nonlinear regression tasks. A hierarchical-optimization problem underpins the learning task: Search in a reproducing kernel Hilbert space (RKHS) for a function that minimizes a sample average ℓ -norm (1 ≤ ≤ 2) error loss defined on data contaminated by noise and outliers, under affine constraints defined as the set of minimizers of a quadratic loss on a finite number of faithful data devoid of noise and outliers (side information). To surmount the computational obstacles inflicted by the choice of loss and the potentially infinite dimensional RKHS, approximations of the ℓ -norm loss, as well as a novel twist of the criterion of approximate linear dependency are devised to keep the computational-complexity footprint of the proposed algorithm bounded over time. Numerical tests on datasets showcase the robust behavior of the advocated framework against different types of outliers, under a low computational load, while satisfying at the same time the affine constraints, in contrast to the state-of-the-art methods which are constraint agnostic.

Nonparametric Bellman Mappings for Reinforcement Learning: Application to Robust Adaptive Filtering

“…Notwithstanding, the LS loss is notoriously sensitive to the presence of outliers [3], where outliers are defined as contaminating data that do not adhere to a nominal data-generation model, and are often viewed as random variables (RVs) with non-Gaussian heavy tailed distributions, e.g., α-stable ones [4,5]. To counter outliers in AdaFilt, non-LS losses, such as the p-norm (2 > p ∈ R ++ ) [6][7][8][9][10][11][12][13] and correntropy [14,15] have been studied (henceforth, R ++ will denote the set of all positive real numbers).…”

“…Proof: See Appendix A. More variations of ( 9) can be generated from (10) by tuning the loss functions L, R appropriately. For example, robust B-Map designs against outliers in sampling can be obtained by letting the ℓ 1 -norm take the place of the quadratic one in (12a) and (14a).…”

“…For example, robust B-Map designs against outliers in sampling can be obtained by letting the ℓ 1 -norm take the place of the quadratic one in (12a) and (14a). Task (10) for general (non)smooth convex L and R can be handled efficiently by [54]. Such designs are deferred to future publications.…”