SOS: Score-based Oversampling for Tabular Data

Abstract

Score-based generative models (SGMs) are a recent breakthrough in generating fake images. SGMs are known to surpass other generative models, e.g., generative adversarial networks (GANs) and variational autoencoders (VAEs). Being inspired by their big success, in this work, we fully customize them for generating fake tabular data. In particular, we are interested in oversampling minor classes since imbalanced classes frequently lead to sub-optimal training outcomes. To our knowledge, we are the first presenting a scorebased tabular data oversampling method. Firstly, we re-design our own score network since we have to process tabular data. Secondly, we propose two options for our generation method: the former is equivalent to a style transfer for tabular data and the latter uses the standard generative policy of SGMs. Lastly, we define a finetuning method, which further enhances the oversampling quality.In our experiments with 6 datasets and 10 baselines, our method outperforms other oversampling methods in all cases. CCS CONCEPTS• Computing methodologies → Artificial intelligence.

STaSy: Score-based Tabular data Synthesis

“…OCT-GAN (Kim et al, 2021) is a generative model design based on neural ODEs. SOS (Kim et al, 2022) proposed a style-transfer-based oversampling method for imbalanced tabular data using SGMs, whose main strategy is converting a major sample to a minor sample. Since its task is not compatible to our task to generate from the scratch, direct comparisons are not possible.…”

“…Since STaSy can be converted to an oversampling method following the design guidance of SOS, we conduct oversampling experiments with STaSy to compare with SOS (Kim et al, 2022). We train, for fair comparison, STaSy w/o fine-tuning for each minor class and generate minority samples to be the same size of the majority class.…”

“…Tabular data synthesis is of non-trivial importance in real-world applications for various reasons: protecting the privacy of original tabular data by releasing fake tabular data (Park et al, 2018;Lee et al, 2021), augmenting the original tabular data with fake data for better training machine learning models (Chawla et al, 2002;Han et al, 2005;He et al, 2008;Kim et al, 2022), and so on. However, it is well-known that tabular data frequently has such peculiar characteristics that deep generative models are not able to synthesize all possible details of the original tabular data (Park et al, 2018;Xu et al, 2019) -given a set of columns in tabular data, columns typically follow unpredictable (multi-modal) distributions and therefore, it is hard to model their joint probability.…”