A Cautionary Note on Using G²(dif) to Assess Relative Model Fit in Categorical Data Analysis

Abstract

The likelihood ratio test statistic G 2 (dif) is widely used for comparing the fit of nested models in categorical data analysis. In large samples, this statistic is distributed as a chi-square with degrees of freedom equal to the difference in degrees of freedom between the tested models, but only if the least restrictive model is correctly specified. Yet, this statistic is often used in applications without assessing the adequacy of the least restrictive model. This may result in incorrect substantive conclusions as the above large sample reference distribution for G 2 (dif) is no longer appropriate. Rather, its large sample distribution will depend on the degree of model misspecification of the least restrictive model. To illustrate this, a simulation study is performed where this statistic is used to compare nested item response theory models under various degrees of misspecification of the least restrictive model. G 2 (dif) was found to be robust only under small model misspecification of the least restrictive model. Consequently, we argue that some indication of the absolute goodness of fit of the least restrictive model is needed before employing G 2 (dif) to assess relative model fit.The two most widely used statistics for assessing the goodness of fit of a model fitted to a contingency table are Pearson's x 2 statistic and the likelihood ratio statistic G 2 . Under the null hypotheses that the tested model holds in the popula- MULTIVARIATE BEHAVIORAL RESEARCH, 41(1),[55][56][57][58][59][60][61][62][63][64]

Estimation of the Gastric Cancer Incidence in Tehran by Two-Source Capture-recapture

“…This can affect the increase of the average incidence age in registered cases reported only by the source of death certificates to the population-based cancer registry system. Incidence in men over women was 1.99, while this ratio is different from country reports, as it has been equal to 2.59, 2.61 and 2.58 in country reports of the years of 2005, 2006 and 2007, respectively and also has been equal to 2.33 in Ardabil's population-based cancer registry report during 2004-2006(Babaei et al, 2009). These ratios show very differences in cancer incidence in male and female subgroups of the population under the coverage of Tehran Metropolitan area Cancer Registration in comparison with other country and regional reports.…”

“…G 2 statistic is among the indices used for comparing the models and selecting the best one in the case of using two or more sources (Maydeu-Olivares and Cai., 2006). Therefore, when three sources are available, researcher calculate G 2 statistic for possible choices of two sources capture-recapture method, and select the model with the lowest G 2 statistic as the most appropriate model.…”

Generalized full-information item bifactor analysis.

“…Using notation from above, let log L A ( β̂ A | Y 1 , …, Y G ) be the maximized log-likelihood of Model A, and let log L B ( β̂ B | Y 1, …, Y G ) be the maximized log-likelihood of Model B. Then under appropriate conditions (Haberman, 1977; see also Maydeu-Olivares & Cai, 2006), minus 2 times the difference of the two log-likelihoods…”

Limited‐information goodness‐of‐fit testing of hierarchical item factor models

“…Such an assessment of relative fit has been a common approach to IRT model selection. Among the ‘appropriate conditions’ for using the likelihood ratio test, however, is that the less constrained model be correctly specified (Maydeu‐Olivares & Cai, 2006), which is a matter of absolute model fit. Here, we use the fit statistics to evaluate how well this condition is met (i.e., the extent to which the bifactor model fits the data).…”