On the detection of outlier clinics in medical and surgical trials: II. Theoretical considerations☆

Abstract

In the conduct of multicenter clinical trials, comparisons among the clinical centers are often made with respect to such measures as incidence of diagnosis of various events, operative mortality, and patients adherence to the treatment regimen. Slippage and outlier tests for statistically identifying clinics that are outliers - that is, truly different from the rest of the clinics - with respect to such measures are described. Critical values obtained for these tests by computer simulation are given for various numbers of clinics, numbers of patients enrolled per clinic, and probabilities (p) of a patient incurring a given event under the null hypothesis of no differences among the clinics. Previously published critical values based on normally distributed response variables are applicable only for p close to 0.5. The slippage and outlier tests are compared with respect to statistical power. Alternative forms of the slippage test based on the angular transformation and on hypergeometric probabilities are considered.

Addressing the Challenge of Assessing Physician-Level Screening Performance: Mammography as an Example

“…While the challenge of demonstrating statistical differences due to low event rate in cancer screening has been recognized in the context of clinical trials [10], [11], [37] and practice-level performance accuracy assessment [6], [15], we extend this cautionary theme to physician-level performance measurement and also provide an intuitive graphical solution to avoid misclassification based on insufficient volume.…”

Monitoring potential adverse event rate differences using data from blinded trials: the canary in the coal mine

“…The possibility of increased risk in some of the individual trials can be investigated by outlier detection methods (e.g. ) for identifying centers whose observed values differ materially from their expectations, given R. Even if heterogeneity appears to be unlikely, an investigator still may decide to look more closely at trials with potentially outlying total event counts.…”