A data generation framework for extremely rare case signals

Abstract

Unlike data augmentation, data generation for extremely rare cases is an approach that can spawn a significant number of high-quality samples based on very few original data. This could be useful in anomaly detection and classification tasks that have the limitation of publicly available datasets for research purposes. Though some other approaches have attempted to solve this problem, such as data augmentation techniques, there was nothing to ensure the characteristics of synthesized samples. Previously, we initiated a framework, called Data Augmentation and Generation for Anomalous Time-series Signals (DAGAT), that was in cooperation with important components: Data Augmentation, Variational Autoencoder (VAE), Data Picker (DP), Signal Fragment Assembler (SFA), and Quality Classifier (QC). And then, an upgraded framework, called An Advanced Data Generation for Anomalous Signals (ADGAS), was introduced to eliminate the limitations of DAGAT; those are uncontrollable outputs and the possibility of bad data included in a training set. By reforming DAGAT architecture, ADGAS achieves a better outcome of generated samples. Nonetheless, ADGAS could be improved through better SFA, DP, and QC. Hence, this paper proposed a Data Generation Framework for Extremely Rare Case Signals. The proposed framework is achievable in generating reliable data for various objectives. We challenged this framework by using the 1D-CNN to serve as the performance evaluator in multi-class anomalous classifications and using the water treatment and water distribution testbed (SWaT and WADI) as the real-world anomaly datasets. The result shows that it surpasses other baseline methods of anomaly data augmentation and data generation techniques.

Development of artificial intelligence for automated measurement of cervical lordosis on lateral radiographs

“…One is data augmentation. Because it is difficult to collect rare cases, data is generated using a generative adversarial network 25 . The other method is to report rare case measurements as outliers.…”

A Dataset of Lost Opportunities. The Case of 63 Developed and Emerging Countries Amid the Covid-19 Pandemic

“…Olonade et al [17], Prayitno et al [18] and Chalongvorachai and Woraratpanya [19] and formed the theoretical basis of the research.…”

A Systematic Review of Rare Events Detection Across Modalities using Machine Learning and Deep Learning

“…The Time Window Slicing (TWS) function trims time series data and isolates anomaly incidents, increasing the sample substance before widening the augmentation possibilities across domains. Once the TWS process is complete, anomaly incident seeds are transformed using upsampling-downsampling, fast Fourier transform, and time series decomposition [60].…”

“…High-level features play a crucial role in enhancing the accuracy and robustness of signal detection. Employing features extraction techniques like principal component analysis (PCA) and independent component analysis (ICA) offers the capability to effectively reduce the dimension of the data while extracting features [60].…”