Synthesis of Gaussian trees with correlation sign ambiguity: An information theoretic approach

Abstract

In latent Gaussian trees the pairwise correlation signs between the variables are intrinsically unrecoverable. Such information is vital since it completely determines the direction in which two variables are associated. In this work, we resort to information theoretical approaches to achieve two fundamental goals: First, we quantify the amount of information loss due to unrecoverable sign information. Second, we show the importance of such information in determining the maximum achievable rate region, in which the observed output vector can be synthesized, given its probability density function. In particular, we model the graphical model as a communication channel and propose a new layered encoding framework to synthesize observed data using upper layer Gaussian inputs and independent Bernoulli correlation sign inputs from each layer. We find the achievable rate region for the rate tuples of multi-layer latent Gaussian messages to synthesize the desired observables. 1 A. Moharrer, and S. Wei are with the school

Successive synthesis of latent Gaussian trees

“…In [10] we showed that the vectors X and B are independent. We argued the intrinsic sign singularity in Gaussian trees is due to the fact that the pairwise correlations ρ xixj ∈ Σ x can be written as (l,k)∈E ρ x l x k , i.e., the product of correlations on the path from x i to x j .…”

“…Hence, roughly speaking, one can carefully change the sign of several correlations of the path, and still maintain the same value for ρ xixj . We showed that if the cardinality of the input vector Y is k, then 2 k minimal Gaussian trees (that only differ in sign of pairwise correlations) may induce the same joint Gaussian density p X [10].…”

“…Here, we aim to minimize the bounds on achievable rates shown in (4) to make our encoding more efficient. Considering the first lower bound in (4), we derived an interesting result in [10] that shows for any Gaussian tree such mutual information value is only a function of given p X (x) ∼ N (0, Σ x ). However, considering the second second inequality in (4), in Theorem 2, whose proof can be found in Appendix A. we show that under Gaussian tree assumption, the lower bound I[Y (l+1) ; Y (l) |B (l) ], l ∈ [0, L − 1] is minimized for homogeneous Bernoulli sign inputs.…”

“…• In our previous work [21], we only characterized the achievable rate regions for output synthesis of the latent Gaussian trees with leaf observables, and with each hidden node only connected to the upper layer inputs. However, in this paper not only we provide a constructive proof for those subclass of Gaussian trees, but also we completely characterize the synthesis scheme to generate the entire statistics of any latent Gaussian tree structure.…”

“…We show that in latent Gaussian trees, we are always having a sign singularity issue [10], which we can exploit to make our synthesis approach more efficient. To clarify, consider the following example.…”