Disentanglement and Local Directions of Variance

Rakowski, Alexander; Lippert, Christoph

doi:10.1007/978-3-030-86523-8_2

Cited by 1 publication

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another line of research focuses on revealing inner mechanisms and inductive biases (Ridgeway 2016) for VAE-based models, and the PCA inductive biases are demystified (Rolinek, Zietlow, and Martius 2019;Zietlow, Rolinek, and Martius 2021;Bao et al 2020) recently. It has been analyzed that the PCA inductive biases induce VAEs to improve local alignment of latent dimensions with principal components of the data (Zietlow, Rolinek, and Martius 2021;Rakowski and Lippert 2021). In this paper, we propose novel geometric inductive biases for unsupervised disentangling that induce the model to capture the global geometric properties of the data manifold, and the model identifiability is proven to be rigorously guaranteed.…”

Section: Introductionmentioning

confidence: 99%

Geometric Inductive Biases for Identifiable Unsupervised Learning of Disentangled Representations

Pan

Niu

Zhang

2023

AAAI

View full text Add to dashboard Cite

The model identifiability is a considerable issue in the unsupervised learning of disentangled representations. The PCA inductive biases revealed recently for unsupervised disentangling in VAE-based models are shown to improve local alignment of latent dimensions with principal components of the data. In this paper, in additional to the PCA inductive biases, we propose novel geometric inductive biases from the manifold perspective for unsupervised disentangling, which induce the model to capture the global geometric properties of the data manifold with guaranteed model identifiability. We also propose a Geometric Disentangling Regularized AutoEncoder (GDRAE) that combines the PCA and the proposed geometric inductive biases in one unified framework. The experimental results show the usefulness of the geometric inductive biases in unsupervised disentangling and the effectiveness of our GDRAE in capturing the geometric inductive biases.

show abstract

Section: Introductionmentioning

confidence: 99%