Boosted adaptive filters

Abstract

We investigate boosted online regression and propose a novel family of regression algorithms with strong theoretical bounds. In addition, we implement several variants of the proposed generic algorithm. We specifically provide theoretical bounds for the performance of our proposed algorithms that hold in a strong mathematical sense. We achieve guaranteed performance improvement over the conventional online regression methods without any statistical assumptions on the desired data or feature vectors. We demonstrate an intrinsic relationship, in terms of boosting, between the adaptive mixture-of-experts and data reuse algorithms. Furthermore, we introduce a boosting algorithm based on random updates that is significantly faster than the conventional boosting methods and other variants of our proposed algorithms while achieving an enhanced performance gain. Hence, the random updates method is specifically applicable to the fast and high dimensional streaming data. Specifically, we investigate Recursive Least Squares (RLS)-based and Least Mean Squares (LMS)-based linear regression algorithms in a mixture-of-experts setting, and provide several variants of these well known adaptation methods. Moreover, we extend the proposed algorithms to other filters. Specifically, we investigate the effect of the proposed algorithms on piecewise linear filters. Furthermore, we provide theoretical bounds for the computational complexity of our proposed algorithms. We demonstrate substantial performance gains in terms of mean square error over the constituent filters through an extensive set of benchmark real data sets and simulated examples.

Joint optimization of linear and nonlinear models for sequential regression

“…As a remedy, different models are used together to capture different patterns in real-life data [21], the most common approach being ensemble or mixture models. Ensemble models combine the predictions of several models and are heavily investigated in different fields such as signal processing [22,14], speech recognition [23], computational intelligence, statistics, and machine learning [15,17]. There are several methods for designing ensembles; one can train individual models independently from each other and then combine them [14], or train individual models in a sequential manner where each model focuses on correcting the errors made by the previous ones [16].…”

A Hybrid CFS Filter and RF-RFE Wrapper-Based Feature Extraction for Enhanced Agricultural Crop Yield Prediction Modeling

“…For evaluating the proposed hybrid feature selection with random forest, the regressor model is instantiated with 550 estimator decision trees and 40 random states. Boosting algorithms are a subclass of ensemble algorithms and one of the most widely used algorithms in data science [42], converting weak learners to strong learners. Gradient boosting [43] sequentially trains several models, and every new model consistently reduces the loss function of the entire procedure utilizing the gradient descendant process.…”

Technique to Adjust Adaptive Digital Filter Coefficients in Residue Number System Based Filters

“…In many applications that use modern methods and algorithms for digital signal processing, adaptive digital filters (ADF) are widely used [3,4], including a digital filter and an adaptation system.…”