A small sample model selection criterion based on Kullback's symmetric divergence

Abstract

The Kullback information criterion KIC is a recently developed tool for statistical model selection [I]. KIC serves as an asymptotically unbiased estimator of a variant of the Kullback symmetric divergence, known also as J-divergence. In this paper a bias correction of the Kullback symmetric information criterion is derived for linear models. The conection is of particular use when the sample size is small or when the number of fitted parameters is of moderate to large fraction of the sample sire. For linear regression models, the corrected method called KICc is an exucrly unhiased estimator of a variant of the Kullback symmetric divergence between the true unknown model and the candidate fitted model. Furthermore KICc is found to provide better model order choice than any other asymptotically efficient methods in an application to autoregressive time series models. measures, it functions as a gauge of model disparity, which is arguably more sensitive than either of its individual component. Following the above reasoning, Cavanaugh [ I ] proposed the Kullback information criterion KIC as an asymptotically unbiased estimate of a variant (within a constant) of the J-divergence between the true unknown model and the fitted approximating model. Motivated by the above developments, we propose a bias corrected version of the KIC for linear regression models. The new criterion is shown to outperform classical criteria in a small sample autoregressive modeling. The remainder of this paper is organized as follows. In section 2 we present a short overview of Kullback's directed divergence, AIC, its corrected version AlCc and KIC. In section 3 we introduce the bias corrected version of KIC. Section 4 presents simulation results for autoregressive model selection. We end up by concluding remarks.

Bias of the corrected KIC for underfitted regression models

“…We find that¨© significantly outperforms the other methods in terms of the quality of selected models as measured by the mean square error of approximation. These results strengthen the case for using¨© instead of¨© as originally recommended in [2].…”

“…In addition, one may hope that by improving the bias property, one will also improve the quality of the selected models. This indeed was the motivation behind the development of the corrected¨© criterion proposed in [2] . © is an exactly unbiased estimator of the Kullback symmetric divergence and not only produces drastic bias reduction, but also greatly improves the model selection in small samples.…”

Influence of disturbances on the control of PC-mouse, goal-directed arm movements

“…Here the approximate Kullback information criterion (AKIC) was implemented which attempts to balance the complexity of the model (number of poles) against how well the model fits the original data. The AKIC has the least bias and best resolution of the available model-selection criteria, see [39] for more details. The number of poles was limited to 25% of the total length of the data sequence in order to avoid overfitting in case of a short data sequence.…”

An adaptive hybrid data fusion based identification of skeletal muscle force with ANFIS and smoothing spline curve fitting

“…The model selection criterion used in this paper is Kullback Information Criterion (KIC) which has shown to perform well for sEMG sensor data fusion [7][8][9][10]. The sum of two directed divergences, which is the measure of the models dissimilarity, is known as Kullback's symmetric or J-divergence [17], as given by Equation (5).…”