Data-Driven Optimization of Directed Information Over Discrete Alphabets

Tsur, Dor; Aharoni, Ziv; Goldfeld, Ziv; Permuter, Haim

doi:10.1109/tit.2023.3335863

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article2

Other1

Relationship

Self Cite0

Independent3

Authors

Journals

Cited by 3 publications

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

On Rate Distortion via Constrained Optimization of Estimated Mutual Information

Tsur,

Huleihel,

Permuter

2024

IEEE Access

View full text Add to dashboard Cite

On Rate Distortion via Constrained Optimization of Estimated Mutual Information

Tsur,

Huleihel,

Permuter

2024

IEEE Access

View full text Add to dashboard Cite

Rate Distortion via Constrained Estimated Mutual Information Minimization

Tsur,

Huleihel,

Permuter

2023

2023 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

Optimizing Distributions for Associated Entropic Vectors via Generative Convolutional Neural Networks

Zhang,

Liu,

Kang

et al. 2024

Entropy

View full text Add to dashboard Cite

The complete characterization of the almost-entropic region yields rate regions for network coding problems. However, this characterization is difficult and open. In this paper, we propose a novel algorithm to determine whether an arbitrary vector in the entropy space is entropic or not, by parameterizing and generating probability mass functions by neural networks. Given a target vector, the algorithm minimizes the normalized distance between the target vector and the generated entropic vector by training the neural network. The algorithm reveals the entropic nature of the target vector, and obtains the underlying distribution, accordingly. The proposed algorithm was further implemented with convolutional neural networks, which naturally fit the structure of joint probability mass functions, and accelerate the algorithm with GPUs. Empirical results demonstrate improved normalized distances and convergence performances compared with prior works. We also conducted optimizations of the Ingleton score and Ingleton violation index, where a new lower bound of the Ingleton violation index was obtained. An inner bound of the almost-entropic region with four random variables was constructed with the proposed method, presenting the current best inner bound measured by the volume ratio. The potential of a computer-aided approach to construct achievable schemes for network coding problems using the proposed method is discussed.

show abstract

Data-Driven Optimization of Directed Information Over Discrete Alphabets

Cited by 3 publications

References 44 publications

On Rate Distortion via Constrained Optimization of Estimated Mutual Information

On Rate Distortion via Constrained Optimization of Estimated Mutual Information

Rate Distortion via Constrained Estimated Mutual Information Minimization

Optimizing Distributions for Associated Entropic Vectors via Generative Convolutional Neural Networks

Contact Info

Product

Resources

About