Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013

Abstract

ObjectiveTo conduct a systematic review and meta-analysis of the equivalence between electronic and paper administration of patient reported outcome measures (PROMs) in studies conducted subsequent to those included in Gwaltney et al’s 2008 review.MethodsA systematic literature review of PROM equivalence studies conducted between 2007 and 2013 identified 1,997 records from which 72 studies met pre-defined inclusion/exclusion criteria. PRO data from each study were extracted, in terms of both correlation coefficients (ICCs, Spearman and Pearson correlations, Kappa statistics) and mean differences (standardized by the standard deviation, SD, and the response scale range). Pooled estimates of correlation and mean difference were estimated. The modifying effects of mode of administration, year of publication, study design, time interval between administrations, mean age of participants and publication type were examined.ResultsFour hundred thirty-five individual correlations were extracted, these correlations being highly variable (I2 = 93.8) but showing generally good equivalence, with ICCs ranging from 0.65 to 0.99 and the pooled correlation coefficient being 0.88 (95 % CI 0.87 to 0.88). Standardised mean differences for 307 studies were small and less variable (I2 = 33.5) with a pooled standardised mean difference of 0.037 (95 % CI 0.031 to 0.042). Average administration mode/platform-specific correlations from 56 studies (61 estimates) had a pooled estimate of 0.88 (95 % CI 0.86 to 0.90) and were still highly variable (I2 = 92.1). Similarly, average platform-specific ICCs from 39 studies (42 estimates) had a pooled estimate of 0.90 (95 % CI 0.88 to 0.92) with an I2 of 91.5. After excluding 20 studies with outlying correlation coefficients (≥3SD from the mean), the I2 was 54.4, with the equivalence still high, the overall pooled correlation coefficient being 0.88 (95 % CI 0.87 to 0.88). Agreement was found to be greater in more recent studies (p < 0.001), in randomized studies compared with non-randomised studies (p < 0.001), in studies with a shorter interval (<1 day) (p < 0.001), and in respondents of mean age 28 to 55 compared with those either younger or older (p < 0.001). In terms of mode/platform, paper vs Interactive Voice Response System (IVRS) comparisons had the lowest pooled agreement and paper vs tablet/touch screen the highest (p < 0.001).ConclusionThe present study supports the conclusion of Gwaltney’s previous meta-analysis showing that PROMs administered on paper are quantitatively comparable with measures administered on an electronic device. It also confirms the ISPOR Taskforce´s conclusion that quantitative equivalence studies are not required for migrations with minor change only. This finding should be reassuring to investigators, regulators and sponsors using questionnaires on electronic devicesafter migration using best practices. Although there is data indicating that migrations with moderate changes produce equivalent instrument versions, hence do not require quantitative equivalence studi...

Psychometric evaluation of self-report outcome measures for prosthetic applications

“…Recent meta-analyses have concluded equivalence of paper and electronic self-report instruments in both healthy individuals and those with a variety of medical conditions. 19,54,55 However, none of the studies included in the meta-analyses targeted (or, to our knowledge, included) people with lower limb loss. This study addresses this gap and contributes evidence to the body of knowledge regarding MoA equivalence.…”

Patient-reported outcome measurement in drug discovery: a tool to improve accuracy and completeness of efficacy and safety data

“…developed on paper but administered by web). However, empiric evidence suggest that there is generally equivalence between different modes of administration [18,19]. If different languages are planned, wellconducted linguistic adaptations should be used or developed, and standards for doing so exist [20].…”

The use of digital technologies to collect patient data in outcomes research

“…Recently, however, evidence supporting the conservation of instrument measurement properties across paper and different devices "Leveraging bring your own device, smartphone sensors and shareables in our studies should be encouraged. It has the potential to significantly reduce the proportion of subjects providing data on paper and limit the associated paper data management processes... Editorial Byrom & Row has been summarized in a number of meta-analyses of published equivalence studies [2,3].…”