Assessing Information Transmission in Data Transformations with the Channel Multivariate Entropy Triangle

Valverde-Albacete, Francisco J.; Peláez-Moreno, Carmen

doi:10.3390/e20070498

Cited by 3 publications

(2 citation statements)

References 28 publications

(130 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Abdallah and Plumbley also proposed an interesting non-negative decomposition, named the binding information (definition 23 [ 34 ]). To quantify and represent the transfer of information from a multivariate source to a multivariate sink of information, Valverde-Albacete and Peláez-Moreno defined a 2-simplex in the multivariate entropic space to represent the information balance of the multivariate transformation [ 35 , 36 ].…”

Section: Introductionmentioning

confidence: 99%

Topological Information Data Analysis

Baudot

Tapia

Bennequin

et al. 2019

Entropy

100

View full text Add to dashboard Cite

This paper presents methods that quantify the structure of statistical interactions within a given data set, and was first used in [58]. It establishes new results on the k-multivariate mutual-informations (I k ) inspired by the topological formulation of Information introduced in [4,63]. In particular we show that the vanishing of all I k for 2 ≤ k ≤ n of n random variables is equivalent to their statistical independence. Pursuing the work of Hu Kuo Ting and Te Sun Han [23,21,22], we show that information functions provide co-ordinates for binary variables, and that they are analytically independent on the probability simplex for any set of finite variables. The maximal positive I k identifies the variables that co-vary the most in the population, whereas the minimal negative I k identifies synergistic clusters and the variables that differentiate-segregate the most the population. Finite data size effects and estimation biases severely constrain the effective computation of the information topology on data, and we provide simple statistical tests for the undersampling bias and the k-dependences following [43]. We give an example of application of these methods to genetic expression and unsupervised cell-type classification. The methods unravel biologically relevant subtypes, with a sample size of 41 genes and with few errors. It establishes generic basic methods to quantify the epigenetic information storage and a unified epigenetic unsupervised learning formalism. We propose that higher-order statistical interactions and non identically distributed variables are constitutive characteristics of biological systems that should be estimated in order to unravel their significant statistical structure and diversity. The topological information data analysis presented here allows to precisely estimate this higher-order structure characteristic of biological systems."When you use the word information, you should rather use the word form"René Thom

show abstract

Section: Introductionmentioning

confidence: 99%

Topological Information Data Analysis

Baudot

Tapia

Bennequin

et al. 2019

Entropy

100

View full text Add to dashboard Cite

show abstract

“…Bukovsky et al [ 6 ] discuss and further develop the recently introduced concept of learning entropy (LE) as a learning-based information measure, which is targeted at real-time novelty detection based on unusual learning efforts. For assessing the quality of data transformations in machine learning, Valverde-Albacete et al [ 7 ] introduce an information-theoretic tool. They analyze performance of the tool for different types of data transformation, among them principal component analysis and independent component analysis.…”

mentioning

confidence: 99%