A computationally efficient estimator for mutual information

Evans, Dafydd

doi:10.1098/rspa.2007.0196

Cited by 35 publications

(47 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The process of mutual information involves the estimation of probability density, which is a computationally expensive process. However, the nonparametric estimator for mutual information, presented in [23], limits the computational complexity to O(NlogN ), where N is the number of observations.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Multivariate mutual information for audio video fusion

Dilpazir

Muhammad

Minhas

et al. 2016

SIViP

View full text Add to dashboard Cite

Information fusion is one of the essential part of distributed wireless sensor networks as well as perceptual user interfaces. Irrelevant and redundant data severely affect the performance of the information fusion process. In this paper, a method based on multivariate mutual information is presented to validate the acceptability of data from two sources (visual and auditory). The audiovisual information is fused to observe the ventriloquism effect to validate the algorithm. Unlike the preceding algorithms, this framework does not require any preprocessing such as automatic face recognition. Moreover, statistical modeling or feature extraction and learning algorithms are not required to extract the maximum information regions. The results for various cases, containing a single speaker as well as a group of speakers, are also presented.

show abstract

Section: Discussionmentioning

confidence: 99%

“…In the literature, there are many algorithms to estimate the probability distribution. However, we use the nonparametric estimators presented in [23]. The approach proposed here is not constrained to any underlying assumption on density (differentiable, parametric, etc.…”

Section: Density Estimationmentioning

confidence: 99%

Multivariate mutual information for audio video fusion

Dilpazir

Muhammad

Minhas

et al. 2016

SIViP

View full text Add to dashboard Cite

show abstract

“…The KNN approach [29], [30] estimates mutual information based on the metric properties of nearest neighbor balls in both the joint and marginal spaces. This approach exploits the method proposed by Kozachenko and Leonenko [31] to estimate the joint and marginal entropies of random variables for computing MI as defined in (3).…”

Section: A Non-parametric K-nearest Neighbor Approachmentioning

confidence: 99%

“…Hence, a trade-off between the sampling and systematic errors should be taken into account for choosing the parameter (which determines the size of the th nearest neighbor ball) appropriately to ensure that the total error is small. The reader is referred to [29], [30] for further information regarding the procedure. In this work, we use the KNN estimator of Kraskov et al [29] where the original implementation of this algorithm can be found in [32].…”

Section: A Non-parametric K-nearest Neighbor Approachmentioning

confidence: 99%

Objective Intelligibility Measures Based on Mutual Information for Speech Subjected to Speech Enhancement Processing

Taghia

Martin

2014

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

We propose a novel method for objective speech intelligibility prediction which can be useful in many application domains such as hearing instruments and forensics. Most objective intelligibility measures available in the literature employ some kind of signal-to-noise ratio (SNR) or a correlation-based comparison between the spectro-temporal representations of clean and processed speech. In this paper, we investigate the speech intelligibility prediction from the viewpoint of information theory and introduce novel objective intelligibility measures based on the estimated mutual information between the temporal envelopes of clean speech and processed speech in the subband domain. Mutual information allows to account for higher order statistics and hence to consider dependencies beyond the conventional second order statistics. Using data from three different listening tests it is shown that the proposed objective intelligibility measures provide promising results for speech intelligibility prediction in different scenarios of speech enhancement where speech is processed by non-linear modification strategies.Index Terms-Mutual information, objective measures, speech intelligibility prediction.

show abstract

“…A very good estimator based on k-nearest neighbor statistics is proposed in [8]. A computationally efficient modification of this method appeared recently in [9]. The estimation of mutual information sets in the broader context of the estimation of information-type measures such as entropy, Kullback-Leibler distance, divergence functionals, Rényi entropy.…”

Section: Introductionmentioning

confidence: 99%

Non–Parametric Estimation of Mutual Information through the Entropy of the Linkage

Giraudo

Sacerdote

Sirovich

2013

Entropy

View full text Add to dashboard Cite

A new, non-parametric and binless estimator for the mutual information of a d-dimensional random vector is proposed. First of all, an equation that links the mutual information to the entropy of a suitable random vector with uniformly distributed components is deduced. When d = 2 this equation reduces to the well known connection between mutual information and entropy of the copula function associated to the original random variables. Hence, the problem of estimating the mutual information of the original random vector is reduced to the estimation of the entropy of a random vector obtained through a multidimensional transformation. The estimator we propose is a two-step method: first estimate the transformation and obtain the transformed sample, then estimate its entropy. The properties of the new estimator are discussed through simulation examples and its performances are compared to those of the best estimators in the literature. The precision of the estimator converges to values of the same order of magnitude of the best estimator tested. However, the new estimator is unbiased even for larger dimensions and smaller sample sizes, while the other tested estimators show a bias in these cases.

show abstract

A computationally efficient estimator for mutual information

Cited by 35 publications

References 10 publications

Multivariate mutual information for audio video fusion

Multivariate mutual information for audio video fusion

Objective Intelligibility Measures Based on Mutual Information for Speech Subjected to Speech Enhancement Processing

Non–Parametric Estimation of Mutual Information through the Entropy of the Linkage

Contact Info

Product

Resources

About