Information-theoretic term weighting schemes for document clustering and classification

DLITE: The Discounted Least Information Theory of Entropy

“…It meets the metric properties of non-negativity, identity of indiscernibles, and symmetry. Additionally, its cube root DLIT E 1 3 satisfies the property of triangular inequality and is a metric distance.…”

“…In our prior work, we proposed the Least Information Theory (LIT) to quantify the amount of entropic difference between two probability distributions [3]. Given probability distributions P and Q of the same variable X, LIT is computed by:…”

“…This is the result of the integral of any logarithm. Research has studied the Least Information Theory (LIT) and shown superior results in applications such as text clustering/classification and information retrieval [3,4,1].…”

“…Because its cube root is a metric distance, DLITE can be regarded as a 3-dimensional volumetric measure in the amount of information. We refer to DLIT E 1 3 as the DLITE distance. This characteristic is similar to that of Jessen-Shannon (JS) Divergence, of which the square root is a metric [6,2].…”

On Triangular Inequality of the Discounted Least Information Theory of Entropy (DLITE)

“…However, KL divergence is not a metric and cannot be used as a symmetric distance measure. In addition, KL is unbounded and has undesirable consequences in practical applications, where extremely large value can dominate the scoring function [4], [5]. Research has proposed the Discounted Least Information Theory (DLITE) as an alternative to mitigate some of these issues [7].…”