This paper considers a problem of distributed hypothesis testing and social learning. Individual nodes in a network receive noisy local (private) observations whose distribution is parameterized by a discrete parameter (hypotheses). The marginals of the joint observation distribution conditioned on each hypothesis are known locally at the nodes, but the true parameter/hypothesis is not known. An update rule is analyzed in which nodes first perform a Bayesian update of their belief (distribution estimate) of each hypothesis based on their local observations, communicate these updates to their neighbors, and then perform a "non-Bayesian" linear consensus using the log-beliefs of their neighbors. Under mild assumptions, we show that the belief of any node on a wrong hypothesis converges to zero exponentially fast, and the exponential rate of learning is characterized by the nodes' influence of the network and the divergences between the observations' distributions. For a broad class of observation statistics which includes distributions with unbounded support such as Gaussian mixtures, we show that rate of rejection of wrong hypothesis satisfies a large deviation principle i.e., the probability of sample paths on which the rate of rejection of wrong hypothesis deviates from the mean rate vanishes exponentially fast and we characterize the rate function in terms of the nodes' influence of the network and the local observation models.This paper was presented in part in [1]- [3]. A. Lalitha and T. Javidi are with the

This paper considers the problem of acquiring an unknown target location (among a finite number of locations) via a sequence of measurements, where each measurement consists of simultaneously probing a group of locations. The resulting observation consists of a sum of an indicator of the target's presence in the probed region, and a zero mean Gaussian noise term whose variance is a function of the measurement vector. An equivalence between the target acquisition problem and channel coding over a binary input additive white Gaussian noise (BAWGN) channel with state and feedback is established.Utilizing this information theoretic perspective, a two-stage adaptive target search strategy based on the sorted Posterior Matching channel coding strategy is proposed. Furthermore, using information theoretic converses, the fundamental limits on the target acquisition rate for adaptive and non-adaptive strategies are characterized. As a corollary to the non-asymptotic upper bound of the expected number of measurements under the proposed two-stage strategy, and to non-asymptotic lower bound of the expected number of measurements for optimal non-adaptive search strategy, a lower bound on the adaptivity gain is obtained. The adaptivity gain is further investigated in different asymptotic regimes of interest.Preliminary versions of this work were presented at the 50th Asilomar Conference on Signals, Systems, and Computers, and at 2017 International Symposium on Information Theory.A. Lalitha N. Ronquillo, and T. Javidi are with the Z n (S n ) is equivalent to channel coding over a binary additive white Gaussian noise (BAWGN) channel with state and feedback (in Section 4.6 [1]). This allows us not only to retrofit the known channel coding schemes based on sorted Posterior Matching (sort PM) [2] as adaptive search strategies, but also to obtain information theoretic converses to characterize fundamental limits on the target acquisition rate under both adaptive and non-adaptive strategies. As a corollary to the non-asymptotic analysis of our sorted Posterior-Matching-based adaptive strategy and our converse for non-adaptive strategy, we obtain a lower bound on the adaptivity gain. A. Our ContributionsOur main results are inspired by the analogy between target acquisition under measurement dependent noise and channel coding with state and feedback. This connection was utilized in [3] under a Bernoulli noise model. In this paper, in Proposition 1, we formalize the connection between our target acquisition problem with Gaussian measurement dependent noise and channel coding over a BAWGN channel with state. Here, the channel state denotes the variance of the measurement dependent noise |S n |δσ 2 . Since feedback codes i.e., adapting the codeword to the past channel outputs, are known to increase the capacity of a channel with state and feedback. This motivates us to use adaptivity when searching, i.e., to utilize past observations {Y 1 , Y 2 , . . . , Y n−1 } when selecting the next measurement vector S n . Furthermore, this informati...

No abstract

No abstract

The maximum type-I and type-II error exponents associated with the newly introduced almost-fixed-length hypothesis testing is characterized. In this class of tests, the decisionmaker declares the true hypothesis almost always after collecting a fixed number of samples n; however in very rare cases with exponentially small probability the decision maker is allowed to collect another set of samples (no more than polynomial in n). This class of hypothesis tests are shown to bridge the gap between the classical hypothesis testing with a fixed sample size and the sequential hypothesis testing, and improve the trade-off between type-I and type-II error exponents.

This paper considers a problem of distributed hypothesis testing and cooperative learning. Individual nodes in a network receive noisy local (private) observations whose distribution is parameterized by a discrete parameter (hypotheses). The conditional distributions are known locally at the nodes, but the true parameter/hypothesis is not known. We consider a social ("non-Bayesian") learning rule from previous literature, in which nodes first perform a Bayesian update of their belief (distribution estimate) of the parameter based on their local observation, communicate these updates to their neighbors, and then perform a "non-Bayesian" linear consensus using the log-beliefs of their neighbors. For this learning rule, we know that under mild assumptions, the belief of any node in any incorrect parameter converges to zero exponentially fast, and the exponential rate of learning is a characterized by the network structure and the divergences between the observations' distributions. Furthermore, any (large) deviation from this nominal rate of rejecting wrong hypothesis is known to occur with an exponentially small probability when the bounded likelihood functions is uniformly bounded. While relaxing the assumption of uniformly bounded log-likelihood functions, we show that the rates of rejecting wrong hypotheses, collectively, satisfy a large deviation principle under a mild technical condition on the log-moment generating functions of log-likelihood random variables.

No abstract

No abstract