Learning Latent Representations Across Multiple Data Domains Using Lifelong VAEGAN

Abstract

The problem of catastrophic forgetting occurs in deep learning models trained on multiple databases in a sequential manner. Recently, generative replay mechanisms (GRM), have been proposed to reproduce previously learned knowledge aiming to reduce the forgetting. However, such approaches lack an appropriate inference model and therefore can not provide latent representations of data. In this paper, we propose a novel lifelong learning approach, namely the Lifelong VAEGAN (L-VAEGAN), which not only induces a powerful generative replay network but also learns meaningful latent representations, benefiting representation learning. L-VAEGAN can allow to automatically embed the information associated with different domains into several clusters in the latent space, while also capturing semantically meaningful shared latent variables, across different data domains. The proposed model supports many downstream tasks that traditional generative replay methods can not, including interpolation and inference across different data domains.

Lifelong Mixture of Variational Autoencoders

“…Recently, there have been some attempts to learn crossdomain representations under the lifelong learning by introducing an environment-dependent mask that specifies a subset of generative factors [16], or by proposing a teacher-student lifelong learning framework [15] and a hybrid model [36] based on Generative Adversarial Nets (GANs) [37] and VAEs. The models proposed in [15], [16], [36] are based on the Generative Replay Mechanisms (GRM) aiming to overcome forgetting. However, these methods suffer from poor performance when considering complex data.…”

InfoVAEGAN: Learning Joint Interpretable Representations by Information Maximization and Maximum Likelihood

“…Lately, the likelihood estimation as a regularization term was shown to stabilize adversarial distribution matching [3]. However, these methods only focus on improving the generation capability and do not design suitable objective functions for inducing disentangled representations.…”

“…Such data sets are assumed to be semantically distinct and to represent different categories of data characteristics. Learning disentangled representations that may capture semantic meaningful information can allow to explicitly edit images and is useful for a variety of tasks [1,2,3]. Enabling disentangled representations can overcome overfitting during the training, leading to better generalization in models, [4].…”

Lifelong Twin Generative Adversarial Networks

“…Moreover, existing GAN based lifelong approaches [11] can not learn inference models, which prevent their usefulness for a wide range of applications. In contrast, VAE based lifelong approaches [12] are able to capture crossdomain representations over several tasks but lead to poor performance when learning databases of high complexity, given that VAEs used as generative replay networks tend to produce rather blurred images. Another category of related works is based on the coupled [13] or dual generative models [14].…”

“…Firstly, LAKD does not require to load previously learnt data samples [7] while its memory size does not change as the number of tasks increases. Secondly, it does not require to preserve the model's parameters or even a snapshot of these after each task switch, as we have in other generative replay methods [12].…”