Are Slepian-Wolf rates necessary for distributed parameter estimation?

Gamal, Mostafa El; Lai, Lifeng

doi:10.1109/allerton.2015.7447151

Cited by 10 publications

(9 citation statements)

References 11 publications

(18 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, it was observed in [4,16] that a standard information-theoretic solution such as Slepian-Wolf coding may not always be the way to go, and [16] mentions the the Körner-Marton scheme in this respect. Further, Shimokawa and Amari [19] point out the possible application of the Körner-Marton scheme to distributed parameter estimation in a similar setting and a similar observation is made in [20]. However, to the best of our knowledge, the present work is the first to propose an actual Körner-Marton-based scheme for distributed hypothesis testing and to analyze its performance.…”

Section: Introductionsupporting

confidence: 68%

Binary distributed hypothesis testing via Körner-Marton coding

Haim¹,

Kochman²

2016

2016 IEEE Information Theory Workshop (ITW)

View full text Add to dashboard Cite

We consider the problem of distributed binary hypothesis testing of two sequences that are generated by an i.i.d.doubly-binary symmetric source. Each sequence is observed by a different terminal. The two hypotheses correspond to different levels of correlation between the two source components, i.e., the crossover probability between the two.The terminals communicate with a decision function via rate-limited noiseless links. We analyze the tradeoff between the exponential decay of the two error probabilities associated with the hypothesis test and the communication rates.We first consider the side-information setting where one encoder is allowed to send the full sequence. For this setting, previous work exploits the fact that a decoding error of the source does not necessarily lead to an erroneous decision upon the hypothesis. We provide improved achievability results by carrying out a tighter analysis of the effect of binning error; the results are also more complete as they cover the full exponent tradeoff and all possible correlations.We then turn to the setting of symmetric rates for which we utilize Körner-Marton coding to generalize the results, with little degradation with respect to the performance with a one-sided constraint (side-information setting).

show abstract

Section: Introductionsupporting

confidence: 68%

Binary distributed hypothesis testing via Körner-Marton coding

Haim¹,

Kochman²

2016

2016 IEEE Information Theory Workshop (ITW)

View full text Add to dashboard Cite

show abstract

“…Zhang et al [11] provided minimax lower bounds for a distributed estimation setting in which all terminals observe samples from the same distribution. El Gamal and Lai [12] showed that Slepian-Wolf rates are not necessary for distributed estimation over finite alphabets.…”

Section: Related Workmentioning

confidence: 99%

Distributed Estimation of Gaussian Correlations

Hadar

Shayevitz

2018

2018 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

We study a distributed estimation problem in which two remotely located parties, Alice and Bob, observe an unlimited number of i.i.d. samples corresponding to two different parts of a random vector. Alice can send k bits on average to Bob, who in turn wants to estimate the cross-correlation matrix between the two parts of the vector. In the case where the parties observe jointly Gaussian scalar random variables with an unknown correlation ρ, we obtain two constructive and simple unbiased estimators attaining a variance of (1 − ρ 2 )/(2k ln 2), which coincides with a known but non-constructive random coding result of Zhang and Berger. We extend our approach to the vector Gaussian case, which has not been treated before, and construct an estimator that is uniformly better than the scalar estimator applied separately to each of the correlations. We then show that the Gaussian performance can essentially be attained even when the distribution is completely unknown. This in particular implies that in the general problem of distributed correlation estimation, the variance can decay at least as O(1/k) with the number of transmitted bits. This behavior, however, is not tight: we give an example of a rich family of distributions for which local samples reveal essentially nothing about the correlations, and where a slightly modified estimator attains a variance of 2 −Ω(k) .

show abstract

“…But it is unlikely to be achievable when the sample sets are distributed to the two processors. A recent paper of El Gamal and Lai [33] studies the problem of decentralized minimum-variance unbiased estimation of W based on observations quantized at the rate of b/n. It is shown that Slepian-Wolf rates are not necessary to achieve the centralized estimation performance, but in their protocol b needs to be proportional to n. The optimal rate region for this decentralized estimation problem is still unknown.…”

Section: B Dependent Sample Setsmentioning

confidence: 99%

Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation

Raginsky

2017

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We derive lower bounds on the Bayes risk in decentralized estimation, where the estimator does not have direct access to the random samples generated conditionally on the random parameter of interest, but only to the data received from local processors that observe the samples. The received data are subject to communication constraints, due to quantization and the noise in the communication channels from the processors to the estimator. We first derive general lower bounds on the Bayes risk using information-theoretic quantities, such as mutual information, information density, small ball probability, and differential entropy. We then apply these lower bounds to the decentralized case, using strong data processing inequalities to quantify the contraction of information due to communication constraints.We treat the cases of a single processor and of multiple processors, where the samples observed by different processors may be conditionally dependent given the parameter, for noninteractive and interactive communication protocols. Our results recover and improve recent lower bounds on the Bayes risk and the minimax risk for certain decentralized estimation problems, where previously only conditionally independent sample sets and noiseless channels have been considered. Moreover, our results provide a general way to quantify the degradation of estimation performance caused by distributing resources to multiple processors, which is only discussed for specific examples in existing works.

show abstract

Are Slepian-Wolf rates necessary for distributed parameter estimation?

Cited by 10 publications

References 11 publications

Binary distributed hypothesis testing via Körner-Marton coding

Binary distributed hypothesis testing via Körner-Marton coding

Distributed Estimation of Gaussian Correlations

Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation

Contact Info

Product

Resources

About