Prediction of general practice licensure/certification examination performance by a measure of ‘questionable’ activities in clinical practice: a retrospective cohort study

Abstract

Purpose ‘Low-value’ clinical care and medical services are ‘questionable’ activities, being more likely to cause harm than good or with disproportionately low benefit relative to cost. This study examined the predictive ability of the QUestionable In Training Clinical Activities Index (QUIT-CAI) for general practice (GP) registrars’ (trainees’) performance in Australian GP Fellowship examinations (licensure/certification examinations for independent GP). Methods The study was nested in ReCEnT, an ongoing cohort study in which Australian GP registrars document their in-consultation clinical practice. Outcome factors in analyses were individual registrars’ scores on the three Fellowship examinations (‘AKT’, ‘KFP’, and ‘OSCE’ examinations) and pass/fail rates during 2012–21. Analyses used univariable and multivariable regression (linear or logistic, as appropriate). The study factor in each analysis was ‘QUIT-CAI score percentage’—the percentage of times a registrar performed a QUIT-CAI clinical activity when ‘at risk’ (i.e. when managing a problem where performing a QUIT-CAI activity was a plausible option). Results A total of 1265, 1145, and 553 registrars sat Applied Knowledge Test, Key Features Problem, and Objective Structured Clinical Exam examinations, respectively. On multivariable analysis, higher QUIT-CAI score percentages (more questionable activities) were significantly associated with poorer Applied Knowledge Test scores (P = .001), poorer Key Features Problem scores (P = .003), and poorer Objective Structured Clinical Exam scores (P = .005). QUIT-CAI score percentages predicted Royal Australian College of General Practitioner exam failure [odds ratio 1.06 (95% CI 1.00, 1.12) per 1% increase in QUIT-CAI, P = .043]. Conclusion Performing questionable clinical activities predicted poorer performance in the summative Fellowship examinations, thereby validating these examinations as measures of actual clinical performance (by our measure of clinical performance, which is relevant for a licensure/certification examination).