Algebraic properties of solutions to common information of Gaussian vectors under sparse graphical constraints

Algebraic Properties of Wyner Common Information Solution under Graphical Constraints

“…In the first of the two subsections of this section, we find the conditions under which CMDFA solution of Σ x recovers the model given by (3) or equivalently speaking, find condtions under which CMDFA solution of Σ x is the rank 1 matrix given by (7). In the other subsection, we show the detailed analysis on the existance and uniqueness of the CMDFA solution of Σ x , when the solution is not a rank 1 matrix.…”

“…It is important to remark that our work is not concerned about the algorithm side of the optimization technique, rather our focus is to characterize and find insights about their solution space. Moharrer and Wei [7] derived CMDFA from a broader class of convex optimization problem and established a relationship between the outcome of these optimization techniques and common information problem [9]. We find the explicit conditions under which the CMDFA solution of Σ x recoves a star structure.…”

“…Σ x being produced by the model in (3) would equivalently mean (6) having a rank 1 solution i.e. Σ x − D being Σ t,N D given by equation (7).…”

“…Secondly, Factor Analysis via heuristic local optimization techniques, often based on the expectation maximization algorithm, were computationally tractable but offered no provable performance guarantees. The third and final type are the convex optimization based methods such as Constrained Minimum Trace Factor Analysis (CMTFA) [5] [6] and CMDFA [7]. The motivation behind CMDFA comes from Wyner's common information C(X 1 , X 2 ) which characterizes the minimum amount of common randomness needed to approximate the joint density between a pair of random variables X 1 and X 2 to be C(X 1 , X 2 ) = min P Y X 1 −Y −X 2 I(X 1 , X 2 ; Y ), 1 where I(X 1 , X 2 ; Y ) is the mutual information between X 1 , X 2 and Y , X 1 −Y −X 2 indicates the conditional independence between X 1 and X 2 given Y , and the joint density function is sought to esnure such conditional independence as well as the given joint density of X 1 and X 2 .…”

Latent Factor Analysis of Gaussian Distributions under Graphical Constraints

“…Secondly, Factor Analysis via heuristic local optimization techniques, often based on the expectation maximization algorithm, were computationally tractable but offered no provable performance guarantees. The third and final type are the convex optimization based methods such as Constrained Minimum Trace Factor Analysis (CMTFA) [5] [] and CMDFA [6]. The motivation behind CMDFA comes from Wyner's common information C(X 1 , X 2 ) which characterizes the minimum amount of common randomness needed to approximate the joint density between a pair of random variables X 1 and X 2 to be C(X 1 , X 2 ) = min P Y X 1 −Y −X 2 I(X 1 , X 2 ; Y ), where I(X 1 , X 2 ; Y ) is the mutual information between X 1 , X 2 and Y , X 1 −Y −X 2 indicates the conditional independence between X 1 and X 2 given Y , and the joint density function is sought to esnure such conditional independence as well as the given joint density of X 1 and X 2 .…”