On the rate of learning in distributed hypothesis testing

Lalitha, Anusha; Javidi, Tara

doi:10.1109/allerton.2015.7446979

Cited by 4 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This assumption allows the noisy observation kernels to have an unbounded support and shown to allow for Gaussian and other practically relevant distributions. Additionally, our work strengthens prior work (even for the case of finite log likelihood ratios in [3]- [6]) in two ways. Firstly, our analysis gives a characterization of the joint rejection rate vector.…”

Section: Discussionsupporting

confidence: 67%

“…This assumption is technical assumption one and relaxes the assumption of bounded support for the likelihood ratio random variable in prior work [1], [3]- [6]. Next, we provide families of distributions which satisfy Assumption 2 even though they might have unbounded support.…”

Section: Assumption 2 For Every Pairmentioning

confidence: 95%

“…In [6], under a new assumption for aperiodic networks we provided tight and matching asymptotic upper and lower bound on the rate of decay of wrong hypothesis rejection rates with small deviations from expected rejection rate. This assumption is less stringent and allows us to consider a much larger family of likelihood functions.…”

Section: Introductionmentioning

confidence: 94%

“…In this paper, under the same assumptions of [6] (i.e., while relaxing the assumption of bounded log-likelihood ratios), we obtain a large deviation principle (LDP), where tight asymptotic matching upper and lower bounds on the rate of decay of the rejection rates that deviate from the expected rate, have been characterized for all deviations. Moreover, the asymptotic decay rate for every deviation is greater than the finite time rate as shown in [6]. Also, in this paper we provide concentration rates for the case where the rate of rejection for every wrong hypothesis is considered simultaneously.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Large deviation analysis for learning rate in distributed hypothesis testing

Lalitha

Javidi

2015

2015 49th Asilomar Conference on Signals, Systems and Computers

Self Cite

View full text Add to dashboard Cite

This paper considers a problem of distributed hypothesis testing and cooperative learning. Individual nodes in a network receive noisy local (private) observations whose distribution is parameterized by a discrete parameter (hypotheses). The conditional distributions are known locally at the nodes, but the true parameter/hypothesis is not known. We consider a social ("non-Bayesian") learning rule from previous literature, in which nodes first perform a Bayesian update of their belief (distribution estimate) of the parameter based on their local observation, communicate these updates to their neighbors, and then perform a "non-Bayesian" linear consensus using the log-beliefs of their neighbors. For this learning rule, we know that under mild assumptions, the belief of any node in any incorrect parameter converges to zero exponentially fast, and the exponential rate of learning is a characterized by the network structure and the divergences between the observations' distributions. Furthermore, any (large) deviation from this nominal rate of rejecting wrong hypothesis is known to occur with an exponentially small probability when the bounded likelihood functions is uniformly bounded. While relaxing the assumption of uniformly bounded log-likelihood functions, we show that the rates of rejecting wrong hypotheses, collectively, satisfy a large deviation principle under a mild technical condition on the log-moment generating functions of log-likelihood random variables.

show abstract

Section: Discussionsupporting

confidence: 67%

Section: Assumption 2 For Every Pairmentioning

confidence: 95%

Section: Introductionmentioning

confidence: 94%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Large deviation analysis for learning rate in distributed hypothesis testing

Lalitha

Javidi

2015

2015 49th Asilomar Conference on Signals, Systems and Computers

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is a technical assumption that it relaxes the assumption of bounded ratios of the likelihood functions in prior work [1], [2], [31], [43]. Next, we provide families of distributions which satisfy Assumption 5 but violate Assumption 4.…”

Section: Large Deviation Analysismentioning

confidence: 99%

Social learning and distributed hypothesis testing

Lalitha

Sarwate

Javidi

2014

2014 IEEE International Symposium on Information Theory

Self Cite

View full text Add to dashboard Cite

Abstract-This paper considers a problem of distributed hypothesis testing and social learning. Individual nodes in a network receive noisy local (private) observations whose distribution is parameterized by a discrete parameter (hypotheses). The marginals of the joint observation distribution conditioned on each hypothesis are known locally at the nodes, but the true parameter/hypothesis is not known. An update rule is analyzed in which nodes first perform a Bayesian update of their belief (distribution estimate) of each hypothesis based on their local observations, communicate these updates to their neighbors, and then perform a "non-Bayesian" linear consensus using the log-beliefs of their neighbors. Under mild assumptions, we show that the belief of any node on a wrong hypothesis converges to zero exponentially fast, and the exponential rate of learning is characterized by the nodes' influence of the network and the divergences between the observations' distributions. For a broad class of observation statistics which includes distributions with unbounded support such as Gaussian mixtures, we show that rate of rejection of wrong hypothesis satisfies a large deviation principle i.e., the probability of sample paths on which the rate of rejection of wrong hypothesis deviates from the mean rate vanishes exponentially fast and we characterize the rate function in terms of the nodes' influence of the network and the local observation models.

show abstract