Development of a clinical prediction model for an ordinal outcome: the World Health Organization Multicentre Study of Clinical Signs and Etiological Agents of Pneumonia, Sepsis and Meningitis in Young Infants

Abstract

This paper describes the methodologies used to develop a prediction model to assist health workers in developing countries in facing one of the most difficult health problems in all parts of the world: the presentation of an acutely ill young infant. Statistical approaches for developing the clinical prediction model faced at least two major difficulties. First, the number of predictor variables, especially clinical signs and symptoms, is very large, necessitating the use of data reduction techniques that are blinded to the outcome. Second, there is no uniquely accepted continuous outcome measure or final binary diagnostic criterion. For example, the diagnosis of neonatal sepsis is ill-defined. Clinical decision makers must identify infants likely to have positive cultures as well as to grade the severity of illness. In the WHO/ARI Young Infant Multicentre Study we have found an ordinal outcome scale made up of a mixture of laboratory and diagnostic markers to have several clinical advantages as well as to increase the power of tests for risk factors. Such a mixed ordinal scale does present statistical challenges because it may violate constant slope assumptions of ordinal regression models. In this paper we develop and validate an ordinal predictive model after choosing a data reduction technique. We show how ordinality of the outcome is checked against each predictor. We describe new but simple techniques for graphically examining residuals from ordinal logistic models to detect problems with variable transformations as well as to detect non-proportional odds and other lack of fit. We examine an alternative type of ordinal logistic model, the continuation ratio model, to determine if it provides a better fit. We find that it does not but that this model is easily modified to allow the regression coefficients to vary with cut-offs of the response variable. Complex terms in this extended model are penalized to allow only as much complexity as the data will support. We approximate the extended continuation ratio model with a model with fewer terms to allow us to draw a nomogram for obtaining various predictions. The model is validated for calibration and discrimination using the bootstrap. We apply much of the modelling strategy described in Harrell, Lee and Mark (Statist. Med. 15, 361-387 (1998)) for survival analysis, adapting it to ordinal logistic regression and further emphasizing penalized maximum likelihood estimation and data reduction.

Development and validation of a prognostic model for predicting 30-day mortality risk in medical patients in emergency department (ED)

“…In this technique, 1000 sub-samples, each with 500 subjects, of the entire sample were repeatedly sampled (with replacement) and analyzed35. Based on the parameters selected from the most parsimonious models, a nomogram using the “rms” library3536 was constructed for predicting 30-day mortality. In a further validation, the predicted probability of 30-day mortality was compared with the observed probability on the development cohort and validation cohort.…”

Effectiveness of the diabetes education and self management for ongoing and newly diagnosed (DESMOND) programme for people with newly diagnosed type 2 diabetes: cluster randomised controlled trial

“…For ordinal outcomes we used an ordinal regression model with proportional odds assumption, adjusted for clusters. 42 To investigate whether changes in illness beliefs are predictive of changes in outcome variable, we carried out multiple regressions with adjustment for age, sex, and baseline value of the outcome variable. Variables were entered in specified sequence, and we report standardised regression weights (β).…”

Evidence-based detection of pulmonary arterial hypertension in systemic sclerosis: the DETECT study

“…The ROC area under the curve (AUC) formed the criterion for assessing the discriminatory ability of a model. Nomograms20 were derived from the two multivariable risk prediction models (see online supplementary text) to allow classification of patients into risk sets for referral to echocardiography (Step 1) and RHC (Step 2). An alternative algorithm with 65% specificity set in the second step was also evaluated, as was the application of the ESC/ERS guidelines to the DETECT population 1.…”