Entropic criterion for model selection

Abstract

Model or variable selection is usually achieved through ranking models according to the increasing order of preference. One of methods is applying Kullback-Leibler distance or relative entropy as a selection criterion. Yet that will raise two questions, why uses this criterion and are there any other criteria. Besides, conventional approaches require a reference prior, which is usually difficult to get. Following the logic of inductive inference proposed by Caticha [1], we show relative entropy to be a unique criterion, which requires no prior information and can be applied to different fields. We examine this criterion by considering a physical problem, simple fluids, and results are promising.

Comprehensive Assessment of Regional Water Usage Efficiency Control Based on Game Theory Weight and a Matter-Element Model

“…The basic principle is based on the degree of variation among the characteristic values of the evaluation indexes; the greater the degree of variation and the smaller the information entropy, the greater is the weight of the corresponding index, and vice versa. This method avoids human subjectivity and thus yields accurate results [36,37]. The specific steps of the method are as follows.…”

Effect of tidal triggering on seismicity in Taiwan revealed by the empirical mode decomposition method

“…First, we needed to determine the probability distributions of amplitudes of IMFs from real signal and noise. Second, we evaluated the relative entropy or Kullback-Leibler distance of these two distributions to determine their difference (Tseng, 2006;Chen et al, 2007). For the probability distributions from real signals, we simply utilized normalized histograms of amplitudes p real i (t).…”

Using Entropy Leads to a Better Understanding of Biological Systems

“…By repeating this same procedure, one can obtain the joint probability distribution of an L-mer nucleotide sequence; (3) apply the entropic criterion [44,45] to determine the preferred sequence length L (the reader is referred to [16] for further details).…”