Evaluation of psychometric properties of patient-reported outcome measures frequently used in narcolepsy randomized controlled trials: a systematic review

Abstract

Study objectives To systematically determine subjective and objective outcome measures used to measure efficacy of narcolepsy interventions in randomized controlled trials (RCTs) in adults and children and assess psychometric properties of patient-reported outcome measures (PROMs) used. Methods We searched bibliographical databases and clinical trial registries for narcolepsy RCTs and extracted objective and subjective outcome measures. If PROMs were used, we searched for psychometric studies conducted in a narcolepsy population using bibliographical databases and appraised using Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) guidelines. Results 80 different outcome measures were used across 100 RCTs. Epworth Sleepiness Scale (ESS) (n=49) and Maintenance of Wakefulness Test (n=47) were the most frequently used outcome measures. We found nineteen validation studies of ten PROMs in narcolepsy populations. There was limited evidence for validity or responsiveness of the ESS; yet sufficient reliability (pooled ICC: 0.81–0.87). Narcolepsy Severity Scale (NSS) had sufficient reliability (pooled ICC: 0.71–0.92) and both adult and pediatric versions had sufficient discriminant validity (treated/untreated). Content validity was only evaluated in pediatric populations for ESS-CHAD and NSS-P and rated inconclusive. Quality of evidence of the psychometric studies for all scales ranged from very low to low. Conclusions Although recognized by regulatory bodies and widely used as primary outcome measures in trials, there is surprisingly little evidence for validity, reliability and responsiveness of PROMs frequently used to assess treatment efficacy in narcolepsy. The field needs to establish patient-centred minimal clinically important difference for the PROMs used in these trials.

Measuring what matters in healthcare: a practical guide to psychometric principles and instrument development

“…Over a decade later, the situation remains essentially unchanged. In a review of 80 outcome measures used for 100 randomized controlled trials in narcolepsy interventions, authors found ‘surprisingly little evidence for the validity, reliability, and responsiveness of [the instrument] frequently used to assess treatment efficacy in narcolepsy’ ( Schokman et al, 2022 ). Unfortunately, this pattern is repeated in various disciplines, across different populations and diagnostic categories.…”

“…The dearth of quality instruments in healthcare clearly indicates to the clinicians, educators, and researchers that it is in the patients’ and health professionals’ best interests to engage with psychometrics and upskill in knowledge in the field. Too often, instruments are developed poorly, leading to all subsequent data and treatment findings based on their data being questionable ( Schokman et al, 2022 ). Instrument development is a dynamic, iterative process involving creativity, theory, statistical evaluation, and dedication from the instrument authors.…”

Cultural adaptation and validation of the Others as Shamer scale: a barrier to effective health-care assessment

“…There is an imperative need to use instruments that have been validated to increase the likelihood that they accurately measure what they are intended to measure (Waschl and Chen, 2022; Ladak et al , 2022). A suitable measurement is one that is supported by published evidence indicating patient acceptability, reliability, validity and responsiveness (Kowalchuk Horn et al , 2012; Schokman et al , 2022).…”