Entropy and Correlation: Some Comments

Kvålseth, Tarald O.

doi:10.1109/tsmc.1987.4309069

Cited by 197 publications

(94 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…where one way relationship exists. Kvålseth (1987) has briefly defined an information-theoretic measure of dependence between two variables in his paper. This measure has several desirable properties.…”

Section: Some Methods Commonly Used To Measure Dependence Between Varmentioning

confidence: 99%

See 1 more Smart Citation

A new estimate of mutual information based measure of dependence between two variables: properties and fast implementation

Jain

Murthy

2015

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

This article proposes a new method to estimate an existing mutual information based dependence measure using histogram density estimates. Finding a suitable bin length for histogram is an open problem. We propose a new way of computing the bin length for histogram using a function of maximum separation between points. The chosen bin length leads to consistent density estimates for histogram method. The values of density thus obtained are used to calculate an estimate of an existing dependence measure. The proposed estimate is named as MIDI (Mutual Information based Dependence Index). Some important properties of MIDI have also been stated.The performance of the proposed method has been compared to generally accepted measures like distance correlation (dcor), Maximal Information Coefficient (MINE) in terms of accuracy and computational complexity with the help of several artificial data sets with different amounts of noise. The proposed method is able to detect many types of relationships between variables, without making any assumption about the functional form of the relationship. The power statistics of proposed method illustrate their effectiveness in detecting non linear relationship. Thus, it is able to achieve generality without a high rate of false positive cases. MIDI is found to work better on a real life data set than competing methods. The proposed method is found to overcome some of the limitations which occur with dcor and MINE. Computationally, MIDI is found to be better than dcor and MINE, in terms of time and memory, making it suitable for large data sets.

show abstract

Section: Some Methods Commonly Used To Measure Dependence Between Varmentioning

confidence: 99%

“…KM 2 has been used by Kraskov and Grassberger (2009) for clustering. Kvålseth (1987) mentions that the value of all three measures is 1 in case of strict one to one association between the variables. However, for KM 1 this is a sufficient condition but not a necessary condition.…”

Section: Existing Mutual Information Based Measuresmentioning

confidence: 99%

A new estimate of mutual information based measure of dependence between two variables: properties and fast implementation

Jain

Murthy

2015

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

show abstract

“…Performance measures: Four standard measurements are used to evaluate the quality of the clusters: purity [14] measures the accuracy of the dominating class in each cluster, normalized mutual information (Normalized MI) [23] is from the information-theoretic perspective and calculates the mutual dependence of the predicted clustering and the ground-truth partitions, Rand index [24] evaluates true positives within clusters and true negatives between clusters and balanced F-measure considers both precision and recall.…”

Section: Binary Tags and Visual Featuresmentioning

confidence: 99%

Discovering Video Clusters from Visual Features and Noisy Tags

Vahdat

Zhou

Mori

2014

Computer Vision – ECCV 2014

View full text Add to dashboard Cite

Abstract. We present an algorithm for automatically clustering tagged videos. Collections of tagged videos are commonplace, however, it is not trivial to discover video clusters therein. Direct methods that operate on visual features ignore the regularly available, valuable source of tag information. Solely clustering videos on these tags is error-prone since the tags are typically noisy. To address these problems, we develop a structured model that considers the interaction between visual features, video tags and video clusters. We model tags from visual features, and correct noisy tags by checking visual appearance consistency. In the end, videos are clustered from the refined tags as well as the visual features. We learn the clustering through a max-margin framework, and demonstrate empirically that this algorithm can produce more accurate clustering results than baseline methods based on tags or visual features, or both. Further, qualitative results verify that the clustering results can discover sub-categories and more specific instances of a given video category.

show abstract

“…Furthermore, measures of one-way association can be expressed in a general form as different normalizations of conditional entropy, while measures of two-way association as different normalizations of mutual information [9].…”

Section: Measuring Attribute Importancementioning

confidence: 99%