Matrix sketching for supervised classification with imbalanced classes

Abstract

The presence of imbalanced classes is more and more common in practical applications and it is known to heavily compromise the learning process. In this paper we propose a new method aimed at addressing this issue in binary supervised classification. Re-balancing the class sizes has turned out to be a fruitful strategy to overcome this problem. Our proposal performs re-balancing through matrix sketching. Matrix sketching is a recently developed data compression technique that is characterized by the property of preserving most of the linear information that is present in the data. Such property is guaranteed by the Johnson-Lindenstrauss’ Lemma (1984) and allows to embed an n-dimensional space into a reduced one without distorting, within an $$\epsilon $$ ϵ -size interval, the distances between any pair of points. We propose to use matrix sketching as an alternative to the standard re-balancing strategies that are based on random under-sampling the majority class or random over-sampling the minority one. We assess the properties of our method when combined with linear discriminant analysis (LDA), classification trees (C4.5) and Support Vector Machines (SVM) on simulated and real data. Results show that sketching can represent a sound alternative to the most widely used rebalancing methods.

Energy Efficient Streaming Time Series Classification with Attentive Power Iteration

“…Numerous research endeavors have been undertaken to derive a concise and updatable representation from observed data, tailored for streaming analysis (Coleman and Shrivastava 2020;Kowshik et al 2021;Jin et al 2021;Chen et al 2022;Diakonikolas et al 2022;Makarychev, Manoj, and Ovsiankin 2022). Among these approaches, matrix sketching has emerged as a viable solution, finding ap-plication in diverse streaming scenarios such as anomaly detection (Huang and Kasiviswanathan 2015), clustering (Yoo, Huang, and Kasiviswanathan 2016), visualization (Fujiwara et al 2019) and non-time series classifications (Falcone, Anderlucci, and Montanari 2022). A matrix sketch is a reducedsize representation of an input matrix that effectively retains its essential characteristics.…”

On randomized sketching algorithms and the Tracy–Widom law

“…Sketching algorithms have been proposed for key linear statistical methods such as low rank matrix approximation, principal components analysis, linear discriminant analysis and ordinary least squares regression (Mahoney 2011;Woodruff 2014;Erichson et al 2016;Falcone et al 2021). Sketching has also been investigated for Bayesian posterior approximation (Bardenet and Maillard 2015;Geppert et al 2017).…”

“…The bounds in Table 1 only give qualitative guidance about the embedding probability. Users will benefit from more prescriptive results in order to choose the sketch size k, and the type of sketch for applications (Grellmann et al 2016;Geppert et al 2017;Ahfock et al 2020;Falcone et al 2021).…”

On randomized sketching algorithms and the Tracy-Widom law

“…The third column refers to the necessary sketch size k to obtain an -subspace embedding for an arbitrary n × d source dataset with at least probability (1 − δ). (Mahoney, 2011;Woodruff, 2014;Erichson et al, 2016;Falcone et al, 2021). Sketching has also been investigated for Bayesian posterior approximation (Bardenet and Maillard, 2015;Geppert et al, 2017).…”