Lossy Source Compression Using Low-Density Generator Matrix Codes: Analysis and Algorithms

Abstract

Abstract-We study the use of low-density generator matrix (LDGM) codes for lossy compression of the Bernoulli symmetric source. First, we establish rigorous upper bounds on the average distortion achieved by check-regular ensemble of LDGM codes under optimal minimum distance source encoding. These bounds establish that the average distortion using such bounded degree families rapidly approaches the Shannon limit as the degrees are increased. Second, we propose a family of message-passing algorithms, ranging from the standard belief propagation algorithm at one extreme to a variant of survey propagation algorithm at the other. When combined with a decimation subroutine and applied to LDGM codes with suitably irregular degree distributions, we show that such a message-passing/decimation algorithm yields distortion very close to the Shannon rate-distortion bound for the binary symmetric source.

Computing global structural balance in large-scale signed social networks

“…In particular, on the social networks analyzed in this work, we show that the chance that a null model has the level of balance of the true networks is essentially equal to zero. For all three networks, the level of balance turns out to be even less than the Shannon bound one obtains developing a rate-distortion theory for the null models (25,36,37). What makes our signed networks so balanced is the skewed distribution of the signs of the edges on the users: Users with a large majority of friends, but also users with a large majority of enemies, are not causing any significant frustration.…”

Lossy source compression of non-uniform binary sources using GQ-LDGM codes

“…Similarly to what done in [5], after each message-passing run we removed at least r m = 1% of the variables (starting from the more biased ones), as well as all the variables with a bias greater or equal than B m = 0.7, until a maximum of r m = 10% of the variables was removed. We also borrowed from [5] the choices of w i = exp(0.05) and w s = exp(0.10).…”

“…DFTq and IDFTq denote q-point Fourier transformation and its inverse, respectively. As a remark, we were able to derive the given formula for [λ to the limited space, the actual derivations were not included; however, one can notice how these message-updating rules appear as an extension of the ones relative to the GF (2) field and derived in [5].…”

“…One interesting possible workaround to achieve convergence in binary source coding with LDGM codes, as suggested in [5], is to (i) run the traditional message-passing algorithm over an augmented code-space where variables can also take a free state, (ii) set the more certain variables to their preferred (nonfree) states, and (iii) iterate this procedure over the decimated code obtained by eliminating the variables that have been set in step (ii), until all variables have been set. Indeed, this approach can be seen as extending both the standard message-passing rules and the survey propagation ones [11]; in addition, it can be simplified as shown in [12].…”

“…In this paper we aim at extending the algorithms described in [5] to the case in which the binary source to be compressed is not uniform. In this case, in fact, a linear binary code whose codewords have a uniform statistics is no longer optimal [1], as happens in the dual case of channel coding for nonsymmetric channels.…”