Variational Autoencoder for Image-Based Augmentation of Eye-Tracking Data

Abstract

Over the past decade, deep learning has achieved unprecedented successes in a diversity of application domains, given large-scale datasets. However, particular domains, such as healthcare, inherently suffer from data paucity and imbalance. Moreover, datasets could be largely inaccessible due to privacy concerns, or lack of data-sharing incentives. Such challenges have attached significance to the application of generative modeling and data augmentation in that domain. In this context, this study explores a machine learning-based approach for generating synthetic eye-tracking data. We explore a novel application of variational autoencoders (VAEs) in this regard. More specifically, a VAE model is trained to generate an image-based representation of the eye-tracking output, so-called scanpaths. Overall, our results validate that the VAE model could generate a plausible output from a limited dataset. Finally, it is empirically demonstrated that such approach could be employed as a mechanism for data augmentation to improve the performance in classification tasks.

Detection of Abnormal Extraocular Muscles in Small Datasets of Computed Tomography Images Using a Three–dimensional Variational Autoencoder: A Pilot Study

“…Accordingly, VAEs attempt to learn the distributions of latent variables based on the mean values and their variances, rather than the deterministic mapping used by conventional AEs. 28,30,31 Compared with two-dimensional (2D) convolutional neural networks, a 3D convolutional neural network model can encode representations of volumetric elds and therefore extract more discriminative features via the 3D spatial information. 17 However, long computational times and a lack of pre-trained models are limitations of 3D convolutional neural network models.…”

Contribution of Synthetic Data Generation towards an Improved Patient Stratification in Palliative Care

“…Deep generative models, such as Variational Autoencoders (VAE) [ 50 ] and Generative Adversarial Networks (GAN) [ 51 , 52 ], play a key role in this. Although VAEs are also widely applied for generative modeling studies, especially with respect to sparse and scarce data in the medical/health domain for images [ 53 , 54 ] and data integration [ 55 ], relatively few examples for tabular data exist [ 56 , 57 , 58 ]. GANs are currently seen as most promising according to the findings of Xu et al [ 57 ].…”

FaceVAE: Generation of a 3D Geometric Object Using Variational Autoencoders

“…The most well-known generative models in unsupervised learning are approximate density-based variational autoencoders (VAE) [24,[27][28][29][30] and implicit density-based generative adversarial nets (GAN) [31][32][33]. Recently, several papers, such as a study on generating an image through Multi-Adversarial Variational Autoencoder (MAVEN) combining GAN and VAE [34], and a study on generating graph data through Conditional VAE and Long Short Term Memory (LSTM) [35], produced results by combining VAE and other models.…”