Alphabet sizes of auxiliary random variables in canonical inner bounds

Jana, Soumya

doi:10.1109/ciss.2009.5054692

Cited by 10 publications

(15 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consider a pair of random variables

and let

be an arbitrary distortion measure defined for arbitrary reconstruction alphabet

. □ Theorem A1 ([ 112 ]) . Let

be the set of all pairs

satisfying

for some mapping

and some joint distributions

.…”

Section: Proofmentioning

confidence: 99%

“…The main ingredient of this proof is a result by Jana [ 112 ] (Lemma 2.2) which provides a tight cardinality bound for the auxiliary random variables in the canonical problems in network information theory (including noisy source coding problem described in Section 2.3 ). Consider a pair of random variables

and let

be an arbitrary distortion measure defined for arbitrary reconstruction alphabet

.…”

Section: Proofmentioning

confidence: 99%

See 1 more Smart Citation

Bottleneck Problems: An Information and Estimation-Theoretic View

Asoodeh

Calmon

2020

Entropy

View full text Add to dashboard Cite

Information bottleneck (IB) and privacy funnel (PF) are two closely related optimization problems which have found applications in machine learning, design of privacy algorithms, capacity problems (e.g., Mrs. Gerber’s Lemma), and strong data processing inequalities, among others. In this work, we first investigate the functional properties of IB and PF through a unified theoretical framework. We then connect them to three information-theoretic coding problems, namely hypothesis testing against independence, noisy source coding, and dependence dilution. Leveraging these connections, we prove a new cardinality bound on the auxiliary variable in IB, making its computation more tractable for discrete random variables. In the second part, we introduce a general family of optimization problems, termed “bottleneck problems”, by replacing mutual information in IB and PF with other notions of mutual information, namely f-information and Arimoto’s mutual information. We then argue that, unlike IB and PF, these problems lead to easily interpretable guarantees in a variety of inference tasks with statistical constraints on accuracy and privacy. While the underlying optimization problems are non-convex, we develop a technique to evaluate bottleneck problems in closed form by equivalently expressing them in terms of lower convex or upper concave envelope of certain functions. By applying this technique to a binary case, we derive closed form expressions for several bottleneck problems.

show abstract

“…Consider a pair of random variables

and let

be an arbitrary distortion measure defined for arbitrary reconstruction alphabet

. □ Theorem A1 ([ 112 ]) . Let

be the set of all pairs

satisfying

for some mapping

and some joint distributions

.…”

Section: Proofmentioning

confidence: 99%

and let

be an arbitrary distortion measure defined for arbitrary reconstruction alphabet

.…”

Section: Proofmentioning

confidence: 99%

Bottleneck Problems: An Information and Estimation-Theoretic View

Asoodeh

Calmon

2020

Entropy

View full text Add to dashboard Cite

show abstract

“…In order to obtain tight cardinality bounds on the auxiliary random variables used throughout this paper, we refer to a recent result by Jana. In [17], the author carefully applies the Caratheodory-Fenchel-Eggleston theorem in order to obtain tight cardinality bounds on the auxiliary random variables in the Berger-Tung inner bound. This result extends the results and techniques employed by Gu and Effros for the Wyner-Ahlswede-Körner problem [22], and by Gu, Jana, and Effros for the Wyner-Ziv problem [23].…”

Section: A Cardinality Bounds On Auxiliary Random Variablesmentioning

confidence: 99%

“…. , y m ) with random vari- Lemma 4 (Lemma 2.2 from [17]). Every extreme point of A corresponds to some choice of auxiliary variables U 1 , .…”

Section: A Cardinality Bounds On Auxiliary Random Variablesmentioning

confidence: 99%

Multiterminal Source Coding Under Logarithmic Loss

Courtade

Weissman

2014

IEEE Trans. Inform. Theory

146

145

View full text Add to dashboard Cite

Abstract-We consider the two-encoder multiterminal source coding problem subject to distortion constraints computed under logarithmic loss. We provide a single-letter description of the achievable rate distortion region for arbitrarily correlated sources with finite alphabets. In doing so, we also give the rate distortion region for the CEO problem under logarithmic loss. Notably, the Berger-Tung inner bound is tight in both settings.

show abstract

“…We first note that the cardinality bounds in the definition of RD i can be imposed without any loss of generality. This is a consequence of [7,Lemma 2.2] and is discussed in detail in the full manuscript [3]. Thus, it suffices to show that (R 1 , R 2 , D 1 , D 2 ) ∈ RD i , ignoring the cardinality constraints.…”

Section: Multiterminal Source Codingmentioning

confidence: 99%