Learning Temporally Causal Latent Processes from General Temporal Data

Abstract

Our goal is to recover time-delayed latent causal variables and identify their relations from measured temporal data. Estimating causally-related latent variables from observations is particularly challenging as the latent variables are not uniquely recoverable in the most general case. In this work, we consider both a nonparametric, nonstationary setting and a parametric setting for the latent processes and propose two provable conditions under which temporally causal latent processes can be identified from their nonlinear mixtures. We propose LEAP, a theoretically-grounded architecture that extends Variational Autoencoders (VAEs) by enforcing our conditions through proper constraints in causal process prior. Experimental results on various data sets demonstrate that temporally causal latent processes are reliably identified from observed variables under different dependency structures and that our approach considerably outperforms baselines that do not leverage history or nonstationarity information. This is one of the first works that successfully recover time-delayed latent processes from nonlinear mixtures without using sparsity or minimality assumptions.

Learning Latent Causal Dynamics

“…The conditional distribution in their work is assumed to be within exponential families to achieve identifiability on the latent space. The most recent literature on nonlinear ICA for time-series is the work of (Yao et al, 2021), which proposed both a nonparametric condition leveraging the nonstionary noise terms, and a linear, parametric condition leveraging the functional form with generalized Laplacian properties of the noise terms.…”

“…Causal discovery aims to identify the underlying structure of the data generation process by exploiting an appropriate class of assumptions (Spirtes et al, 1993;. Although distribution changes are not desired by time-series models, these changes, serving as a "soft" way of intervention, have proved to greatly improve the identifiability results for learning the latent causal structure (Yao et al, 2021;Bengio et al, 2019;Ke et al, 2019). For this reason, a natural solution is to first exploit the distribution changes to identify the latent causal dynamics, and then use the uncovered structure to correct the model under changes.…”

“…However, most existing work that establishes identifiability results considers either stationary independent sources such as PCL (Hyvarinen & Morioka, 2017), SlowVAE (Klindt et al, 2020) or under lin-ear transition assumptions such as SlowVAE (Klindt et al, 2020) and SNICA (Hälvä et al, 2021), or with certain structure such as Markov properties in HM-NLICA (Hälvä & Hyvarinen, 2020). LEAP (Yao et al, 2021), which is the closest work to ours, has established the identifiability of the nonparametric latent temporal processes in certain nonstationary cases, under the condition that the distribution of the noise terms of the latent processes vary across segments.…”

“…Our theory can be differentiated from the prior work in four aspects as summarized in Table 1: 1. We consider a very general nonstationary case which includes changing causal dynamics such as changes in the influencing strength or switching some edges off (Li et al, 2020), nonstationary noise distributions across domains (Yao et al, 2021), and domain shifts in observation (Sahoo et al, 2021), etc., as special cases;…”

“…The identifiability results under this setting are appealing to real-world data where whether and how the distribution changes are unknown. Our identifiability condition is generally much weaker than the ones in (Yao et al, 2021) because we further benefit from distribution changes under fixed causal dynamics, while LEAP assumes all latent processes are changed across contexts.…”