TTS-GAN: A Transformer-based Time-Series Generative Adversarial Network

Abstract

Signal measurements appearing in the form of time series are one of the most common types of data used in medical machine learning applications. However, such datasets are often small, making the training of deep neural network architectures ineffective. For time-series, the suite of data augmentation tricks we can use to expand the size of the dataset is limited by the need to maintain the basic properties of the signal. Data generated by a Generative Adversarial Network (GAN) can be utilized as another data augmentation tool. RNN-based GANs suffer from the fact that they cannot effectively model long sequences of data points with irregular temporal relations. To tackle these problems, we introduce TTS-GAN, a transformer-based GAN which can successfully generate realistic synthetic time-series data sequences of arbitrary length, similar to the real ones. Both the generator and discriminator networks of the GAN model are built using a pure transformer encoder architecture. We use visualizations and dimensionality reduction techniques to demonstrate the similarity of real and generated time-series data. We also compare the quality of our generated data with the best existing alternative, which is an RNN-based time-series GAN.

Synthetic Subject Generation with Coupled Coherent Time Series Data

“…In this context, Hernandez et al reviewed the SDG approaches proposed as an alternative to anonymization techniques for health domain applications [6]. Furthermore, there are also studies in which STSG has been researched and used [7][8][9][10][11][12][13][14].…”

“…In 2021, Hyun et al [13] proposed NeuralProphet, a neural network variation in the forecasting tool Prophet [17], as a method for STSG to create synthetic diabetic foot patients. In 2022, Li et al [14] presented the transformer-based time-series GAN (TTS-GAN) based on a transformer-encoder architecture.…”

Visually Evaluating Generative Adversarial Networks Using Itself under Multivariate Time Series

“…To illustrate how our method works, the transformer-based time series GANs (ttsGANs) [15] is used because of its better results under PCA visualizations. In addition, the model is trained on the 1 GV100 32G with 4 hours.…”

“…We adopt the same datasets used in the ttsGANs paper [15], the UniMiB dataset [11], where two categories are selected: Jumping and Running because the effect of ttsGAN is different under the two categories. Thus, it can be observed how normality metrics and χ 2 visualization would change while results change from good to bad.…”

“…An obstacle to answering this question was the notorious problem of how to effectively reduce the dimension of MTS [17] [18]. Principle Component Analysis (PCA) is, in most papers, used to reduce the dimension of MTS into two dimensions before plotting the 2D visualization [24] [15]. Otherwise, down-streaming tasks, such as MTS classification tasks, are used to verify whether the generated data is good or not [24] [15].…”