Are responders to patient health surveys representative of those invited to participate? An analysis of the Patient-Reported Outcome Measures Pilot from the Australian Orthopaedic Association National Joint Replacement Registry

Abstract

Background Patient-reported outcome measures (PROMs) are commonly used to evaluate surgical outcome in patients undergoing joint replacement surgery, however routine collection from the target population is often incomplete. Representative samples are required to allow inference from the sample to the population. Although higher capture rates are desired, the extent to which this improves the representativeness of the sample is not known. We aimed to measure the representativeness of data collected using an electronic PROMs capture system with or without telephone call follow up, and any differences in PROMS reporting between electronic and telephone call follow up. Methods Data from a pilot PROMs program within a large national joint replacement registry were examined. Telephone call follow up was used for people that failed to respond electronically. Data were collected pre-operatively and at 6 months post-operatively. Responding groups (either electronic only or electronic plus telephone call follow up) were compared to non-responders based on patient characteristics (joint replaced, bilaterality, age, sex, American Society of Anesthesiologist (ASA) score and Body Mass Index (BMI)) using chi squared test or ANOVA, and PROMs for the two responder groups were compared using generalised linear models adjusted for age and sex. The analysis was restricted to those undergoing primary elective hip, knee or shoulder replacement for osteoarthritis. Results Pre-operatively, 73.2% of patients responded electronically and telephone follow-up of non-responders increased this to 91.4%. Pre-operatively, patients responding electronically, compared to all others, were on average younger, more likely to be female, and healthier (lower ASA score). Similar differences were found when telephone follow up was included in the responding group. There were little (if any) differences in the post-operative comparisons, where electronic responders were on average one year younger and were more likely to have a lower ASA score compared to those not responding electronically, but there was no significant difference in sex or BMI. PROMs were similar between those reporting electronically and those reporting by telephone. Conclusion Patients undergoing total joint replacement who provide direct electronic PROMs data are younger, healthier and more likely to be female than non-responders, but these differences are small, particularly for post-operative data collection. The addition of telephone call follow up to electronic contact does not provide a more representative sample. Electronic-only follow up of patients undergoing joint replacement provides a satisfactory representation of the population invited to participate.

Robotic‐arm assisted total knee arthroplasty has a learning curve of 16 cases and increased operative time of 12 min

“…Despite this, comparisons could be made with outcomes from other TKA groups. PROM response rates were relatively low at times, however 60% response is typical for PROMs without active follow‐up, and was not expected to introduce significant sample bias 34 . Second, long‐leg radiographs were unavailable for verification of post‐surgical component placements.…”

“…PROM response rates were relatively low at times, however 60% response is typical for PROMs without active follow‐up, and was not expected to introduce significant sample bias. 34 Second, long‐leg radiographs were unavailable for verification of post‐surgical component placements. Other studies have validated the accuracy of the CT‐based measurements recorded by the MAKO system, 3 , 20 providing a degree of confidence in these measurements.…”

Association between patient factors and hospital completeness of a patient-reported outcome measures program in joint arthroplasty, a cohort study

“…Any association between post-surgical outcome and response rate would suggest that post-operative retention of patients recruited pre-operatively may be important, whereas initial (pre-operative) recruitment rates may be less important. This may influence the selection of followup timing, as response rates are known to decrease over time [14]. The study findings may not be generalisable to other populations.…”

“…A previous report from the AOANJRR showed that, among patients registered in the PROMs program, representativeness did not change by a large amount with varying patient response rates [10]. A description of the relative response rates between electronic and telephone contact has been previously published showing that responders were more likely to be younger, female and healthier but that these differences were mainly seen pre-operatively rather than post-operatively, and that the addition of telephone follow up did not change the representativeness [14]. Since completing an analysis of the pilot PROMs program, the AOANJRR has now stopped using telephone follow up as the cost does not justify better completeness without significant gains in representativeness.…”

Equity-Driven Implementation of Patient-Reported Outcome Measures in Musculoskeletal Care

“…For example, electronic non-portal-based applications (e.g., SMS [short message service] text messaging 53 ) can be evaluated in subpopulations experiencing access barriers. However, the return on investment needs to be considered for resource-intensive strategies (e.g., patient navigators) that are used to obtain more representative information, particularly for institutions that are already overburdened 54 .…”