The Distributed Karhunen–Loève Transform

Abstract

Abstract-The Karhunen-Loève transform (KLT) is a key element of many signal processing and communication tasks. Many recent applications involve distributed signal processing, where it is not generally possible to apply the KLT to the entire signal; rather, the KLT must be approximated in a distributed fashion. This paper investigates such distributed approaches to the KLT, where several distributed terminals observe disjoint subsets of a random vector.We introduce several versions of the distributed KLT. First, a local KLT is introduced, which is the optimal solution for a given terminal, assuming all else is fixed. This local KLT is different and in general improves upon the marginal KLT which simply ignores other terminals. Both optimal approximation and compression using this local KLT are derived. Two important special cases are studied in detail, namely, the partial observation KLT which has access to a subset of variables, but aims at reconstructing them all, and the conditional KLT which has access to side information at the decoder. We focus on the jointly Gaussian case, with known correlation structure, and on approximation and compression problems. Then, the distributed KLT is addressed by considering local KLTs in turn at the various terminals, leading to an iterative algorithm which is locally convergent, sometimes reaching a global optimum, depending on the overall correlation structure. For compression, it is shown that the classical distributed source coding techniques admit a natural transform coding interpretation, the transform being the distributed KLT. Examples throughout illustrate the performance of the proposed distributed KLT. This distributed transform has potential applications in sensor networks, distributed image databases, hyper-spectral imagery, and data fusion.

Distributed static linear Gaussian models using consensus

“…They focus on saving part of the multi-hop communication cost by either local computations (Kargupta, Huang, Sivakumar, & Johnson, 2001) or aggregation services (Bai, Chan, & Luk, 2005;Le Borgne, Raybaud, & Bontempi, 2008;Qi & Wang, 2004), but they still rely on a fusion center for merging the local results. In the context of distributed compression and source coding, Gastpar, Dragotti, and Vetterli (2006) proposed a distributed Karhunen-Loéve transform which is posed as an optimization problem, where convergence to the global optimum is, in general, not assured. Our two consensus-based distributed PCA algorithms (Valcarcel Macua, Belanovic, & Zazo, 2010) outperform all the above because they both guarantee convergence, with no fusion center, just by local neighborhood communications.…”

A necessary and sufficient condition for transform optimality in source coding

“…The optimal orthogonal transform for source separation of a Gaussian vector scale mixture is KLT, when the contrast function is the divergence-based cost of (20).…”

Universal distributed sensing via random projections

“…A number of distributed coding algorithms have been developed that involve collaboration amongst sensors [5,6]. Any collaboration, however, involves inter-sensor communication overhead that can significantly affect the power consumption of the participating nodes.…”