A Comparison of Goodness-of-Fit Tests for the Logistic Regression Model

Abstract

Recent work has shown that there may be disadvantages in the use of the chi-square-like goodness-of-fit tests for the logistic regression model proposed by Hosmer and Lemeshow that use fixed groups of the estimated probabilities. A particular concern with these grouping strategies based on estimated probabilities, fitted values, is that groups may contain subjects with widely different values of the covariates. It is possible to demonstrate situations where one set of fixed groups shows the model fits while the test rejects fit using a different set of fixed groups. We compare the performance by simulation of these tests to tests based on smoothed residuals proposed by le Cessie and Van Houwelingen and Royston, a score test for an extended logistic regression model proposed by Stukel, the Pearson chi-square and the unweighted residual sum-of-squares. These simulations demonstrate that all but one of Royston's tests have the correct size. An examination of the performance of the tests when the correct model has a quadratic term but a model containing only the linear term has been fit shows that the Pearson chi-square, the unweighted sum-of-squares, the Hosmer-Lemeshow decile of risk, the smoothed residual sum-of-squares and Stukel's score test, have power exceeding 50 per cent to detect moderate departures from linearity when the sample size is 100 and have power over 90 per cent for these same alternatives for samples of size 500. All tests had no power when the correct model had an interaction between a dichotomous and continuous covariate but only the continuous covariate model was fit. Power to detect an incorrectly specified link was poor for samples of size 100. For samples of size 500 Stukel's score test had the best power but it only exceeded 50 per cent to detect an asymmetric link function. The power of the unweighted sum-of-squares test to detect an incorrectly specified link function was slightly less than Stukel's score test. We illustrate the tests within the context of a model for factors associated with low birth weight.

Impact of stroke mechanism in acute basilar occlusion with reperfusion therapy

“…Using the likelihood ratio statistics, a model with the lowest model chi‐square was accepted. Pearson chi‐square, deviance, and Hosmer‐Lemeshow tests were evaluated to assess the goodness of fit 23. Model sensitivity and specificity to predict underlying stroke mechanism were also calculated.…”

Clinical factors predictive of PCR positive in pandemic H1N1 2009 influenza virus infection

“…Analytical results were presented as crude ORs, adjusted ORs, and 95% confidence intervals (CIs). The goodness‐of‐fit of the final model was evaluated using Hosmer–Lemeshow statistics 6 . To evaluate the discriminatory power or accuracy of the model, c statistics or the area under the receiver that was operating characteristic curve was examined 7 .…”

Predicting six-month mortality of patients with traumatic brain injury: usefulness of common intensive care severity scores

“…Therefore, scoring systems that enable quantification of severity of illness are paramount for the evaluation of quality of intensive care [1-4]. Moreover, precise data on severity of illness and the accompanying risk of death are essential in clinical studies [5,6]. However, a scoring system and its associated risk prediction model is useful only if it demonstrates both good calibration and discrimination [7-10].…”