Clinicians’ ratings of the Health of the Nation Outcome Scales (HoNOS) show sensitivity/bias that may affect patients’ progress: analyses of routine administrative data

Abstract

Objectives(1) To estimate clinician sensitivity/bias in rating the HoNOS. (2) To test if high or low clinician sensitivity determines slower resolution of patients’ problems or earlier inpatient admission.DesignThe primary analysis used many-facet Item Response Theory to construct a multi-level Graded Response Model that teased apart clinician sensitivity/bias from the severity of patients’ problems in routine HoNOS records. Secondary analyses then tested if patients’ outcomes depend on their clinicians’ sensitivity/bias.Outcome measuresThe outcome measures were (1) overall differences in sensitivity/bias between (a) individual clinicians and (b) different Community Mental Health Teams (CMHTs); (2) clinical outcomes, comprising (a) the rate of resolution of patients’ problems and (b) the dependence of the time to inpatient admission on clinician sensitivity/bias.SettingAll archival electronic HoNOS records for all new referrals to all CMHTs providing mental health services in secondary care in a New Zealand District Health Board during 2007-2015.ParticipantsThe initial sample comprised 2170 adults of working age who received 5459 HoNOS assessments from 186 clinicians. From these initial data, I derived an opportunistic, connected, bipartite, longitudinal network, in which (i) every patient received HoNOS ratings from 2 or more clinicians and (ii) every clinician assessed more than 5 patients. The bipartite network comprised 88 clinicians and 778 patients; 112 patients underwent later inpatient admission.ResultsSensitivity/bias differed importantly between individual clinicians and CMHTs. Patients whose clinicians had more extreme sensitivity/bias showed slower resolution of their problems and earlier inpatient admission.ConclusionsRaw HoNOS ratings reflect the sensitivity/bias of clinicians almost as much as the severity of patients’ problems. Additionally, low or high clinician sensitivity can adversely affect patients’ outcomes. Hence, the HoNOS’s main value may be to measure clinician sensitivity. Accounting for clinician sensitivity could enable the HoNOS to fulfil its goal of improving mental health services.Strengths and limitations of the studyThe study derived a connected network of clinicians and patients that approximates a rational design for estimating clinicians’ sensitivity/bias.The opportunistic network sample was atypical, with chronic patients and experienced clinicians – so the study may under-estimate clinician bias.The study’s statistical methods were appropriate to the ordinal nature of HoNOS ratings.The study used earlier estimates of clinician sensitivity/bias to predict later outcomes – so that effects of clinician sensitivity/bias on outcomes may be causalThe study assumed that all HoNOS items tap a single dimension of the severity of patients’ problems.