Learning Extremal Representations with Deep Archetypal Analysis

Abstract

Archetypes represent extreme manifestations of a population with respect to specific characteristic traits or features. In linear feature space, archetypes approximate the data convex hull allowing all data points to be expressed as convex mixtures of archetypes. As mixing of archetypes is performed directly on the input data, linear Archetypal Analysis requires additivity of the input, which is a strong assumption unlikely to hold e.g. in case of image data. To address this problem, we propose learning an appropriate latent feature space while simultaneously identifying suitable archetypes. We thus introduce a generative formulation of the linear archetype model, parameterized by neural networks. By introducing the distance-dependent archetype loss, the linear archetype model can be integrated into the latent space of a deep variational information bottleneck and an optimal representation, together with the archetypes, can be learned end-to-end. Moreover, the information bottleneck framework allows for a natural incorporation of arbitrarily complex side information during training. As a consequence, learned archetypes become easily interpretable as they derive their meaning directly from the included side information. Applicability of the proposed method is demonstrated by exploring archetypes of female facial expressions while using multi-rater based emotion scores of these expressions as side information. A second application illustrates the exploration of the chemical space of small organic molecules. By using different kinds of side information we demonstrate how identified archetypes, along with their interpretation, largely depend on the side information provided.

Archetypal Analysis and DEA Model, Their Application on Financial Data and Visualization with PHATE

“…It has found widespread applications in recent years. Archetypes are derived as solutions to an iterative nonlinear optimization algorithm that minimizes RSS (residual sum of squares), i.e., average distance between observations and archetypes [ 2 , 3 , 4 ].…”

“…The optimization procedure concerns also the hyperparameter , the number of archetypes. Thanks to the development of computational tools, archetypal analysis has gained much attention in recent years [ 4 , 5 , 6 ].…”

“…There is another approach that can be used for the description of firms—archetypal analysis, introduced in 1994 by Breiman and Cutler and described in [ 2 ] and newly applied in [ 3 , 4 , 5 , 6 ]. While segmentation is the grouping of similar objects, the analysis of archetypes focuses on the identification of companies (archetypes) spanning the indicator space, i.e., the space between extreme objects treated as a frame.…”

Neural ADMIXTURE: rapid population clustering with autoencoders

“…Several autoencoder architectures similar to our work have been previously proposed. Some examples include the Dirichlet Variational Autoencoder [8], Deep Archetypal Analysis (DeepAA) [9], and Genotype Convolutional Autoencoder (GCAE) [10]. Such networks encode each sample as a point within a convex hull, or as a set of proportions and probabilities.…”

Learning Conditional Invariance through Cycle Consistency

“…However, it was shown that inductive bias on the dataset and learning approach is necessary to obtain disentanglement [25]. Inductive biases allow us to express assumptions about the generative process and to prioritise different solutions not only in terms of disentanglement [5,13,21,35,44], but also in terms of constrained latent space structures [15,16], preservation of causal relationships [40], or interpretability [45].…”