Bounds for network information flow with correlated sources

Thakor, Satyajit; Chan, Terence; Grant, Alex

doi:10.1109/ausctw.2011.5728735

“…This paper further extends [3] to the case of random vector. Theorem 2 (Random Vector Case): Suppose (X 1 , .…”

Section: Resultsmentioning

confidence: 93%

“…Theorem 1 (Random Scalar Case [3]): Suppose X is an m-ary random variable taking values from the set {1, . .…”

Section: Resultsmentioning

confidence: 99%

“…Using auxiliary random variables to characterise source correlation was first proposed in [3]. To illustrate the idea, consider a set of random variables (X 1 , .…”

Section: Resultsmentioning

confidence: 99%

“…Following [2], a linear programming outer bound was developed for dependent sources [3]. This bound is specified by a set of information inequalities and equalities, and source dependence is represented by the entropy function…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Characterising correlation via entropy functions

Thakor

¹

,

Chan

²

,

Grant

³

2013

2013 IEEE Information Theory Workshop (ITW)

Self Cite

View full text Add to dashboard Cite

Characterising the capacity region for a network can be extremely difficult. Even with independent sources, determining the capacity region can be as hard as the open problem of characterising all information inequalities. The majority of computable outer bounds in the literature are relaxations of the Linear Programming bound which involves entropy functions of random variables related to the sources and link messages. When sources are not independent, the problem is even more complicated. Extension of Linear Programming bounds to networks with correlated sources is largely open. Source dependence is usually specified via a joint probability distribution, and one of the main challenges in extending linear program bounds is the difficulty (or impossibility) of characterising arbitrary dependencies via entropy functions. This paper tackles the problem by answering the question of how well entropy functions can characterise correlation among sources. We show that by using carefully chosen auxiliary random variables, the characterisation can be fairly "accurate".

show abstract

“…Grant is with Myriota Pty Ltd. The material in this paper was presented in part at the Australian Communication Theory Workshop, Melbourne, Australia, 2011 [1], Information Theory Workshop, Seville, Spain, 2013 [2] and accepted in International Symposium on Information Theory and its Applications, California, USA, 2016 [3]. T. Chan is supported in part by the Australian Research Council under Discovery Projects DP150103658.…”

Section: Introductionmentioning

confidence: 99%

Capacity Bounds for Networks With Correlated Sources and Characterisation of Distributions by Entropies

Thakor

¹

,

Chan

²

,

Grant³

2017

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

Characterising the capacity region for a network can be extremely difficult. Even with independent sources, determining the capacity region can be as hard as the open problem of characterising all information inequalities. The majority of computable outer bounds in the literature are relaxations of the Linear Programming bound which involves entropy functions of random variables related to the sources and link messages. When sources are not independent, the problem is even more complicated. Extension of Linear Programming bounds to networks with correlated sources is largely open. Source dependence is usually specified via a joint probability distribution, and one of the main challenges in extending linear program bounds is the difficulty (or impossibility) of characterising arbitrary dependencies via entropy functions. This paper tackles the problem by answering the question of how well entropy functions can characterise correlation among sources. We show that by using carefully chosen auxiliary random variables, the characterisation can be fairly "accurate". Using such auxiliary random variables we also give implicit and explicit outer bounds on the capacity of networks with correlated sources. The characterisation of correlation or joint distribution via Shannon entropy functions is also applicable to other information measures such as Rényi entropy and Tsallis entropy.

show abstract