Are adequate PROMs used as outcomes in randomized controlled trials? an analysis of 54 trials

Abstract

Results by patient‐reported outcome measures (PROMs) from randomized controlled trials (RCTs) in musculoskeletal research often influence healthcare strategies. We aimed to evaluate to which extent these RCTs use adequate PROMs, and how this influences the results and conclusions. We identified RCTs of sports research relevance with PROMs as primary outcomes published in 13 preselected journals between January 1, 2008, and November 1, 2019; all journals regularly publish results from musculoskeletal research. Five journals have a high impact factor (>15), and eight with lower impact factors are widely read journals. It was assessed whether the RCTs had used PROMs with high content validity and whether the most adequate PROMs were used (ie, the most well developed and well validated for the patients enrolled in the study). We registered journal impact factor, year of publication, existence of a registered protocol, and whether the study showed significant difference between interventions. A total of 54 RCTs with 56 primary outcomes comprising 26 different PROMs were identified. For 13 RCTs (24%), a protocol was not published. In only 24 of RCTs (44%), the most appropriate PROM had been used as primary outcome, independent of a registered protocol, ranking of the journal, and year of publication. In seven cases, PROMs were used to evaluate a condition that they had not been developed for. RCTs that used the most adequate PROM showed significantly more often (46%) difference in outcomes in contrast to RCTs that used inadequate PROMs (22%) (P = 0.0483). In conclusion, in the majority of RCTs, the most adequate PROM had not been used. Studies, in which the most adequate PROM had been used as outcome, were significantly more likely to show significant difference between interventions. The extent to which protocols were not available was surprisingly high. Journals should request that adequate PROMs are used in RCTs, and if this is not the case that it is discussed how it might influence the results and conclusions. Likewise, it should be requested that a protocol is published or registered.

KOOS-Child exhibits inadequate structural validity in a cohort of paediatric patients with ACL deficiency

“…Despite their irrelevance, all items were retained in KOOS-Child v. 1.0, although three items (A8, A11, A15) were subsequently removed due to ceiling effects 1. Ceiling effect is a well-known consequence of irrelevant items, and it reduces responsiveness and induces type 2 errors 21 27. In our study, substantial ceiling effects were observed for all items in the ADL subscale (table 5).…”

“…Therefore, the impact of the inadequacies visualised by the current analyses suggests a high risk of measurement error in clinical trials using KOOS-Child as an outcome, particularly type 2 errors (ie, false negatives). In clinical trials, a significant difference between treatment groups is found twice as often in studies that use the most adequate patient-reported outcome measure (PROM) compared studies using a less adequate PROM 21. This means, that although all subscales were responsive, they have a poor ability to detect a true difference between the two groups when interventions are compared.…”

Are PROMs used adequately in sports research? An analysis of 54 randomized controlled trials with PROMs as endpoint

“…In 12 of 13 selected journals, we were able to identify one or more relevant RCTs (Table 1). A total of 56 PROMs had been used in 54 different RCTs (described in detail in Table 1 in 5 ). Of these PROMs, 42 were used in RCTs with a published protocol, 21 were from studies that were published in a high‐impact journal, and 31 from recent RCTs.…”

“…A total of 54 studies were included, and 56 PROMs were eligible for assessment. A flowchart of the selection process is shown in Ref 5 …”

A catalogue of PROMs in sports science: Quality assessment of PROM development and validation

“…Additionally, 13 PROMs that had been used only once or twice, but were the only PROMs for a particular condition, were also included in the selection. Furthermore, six PROMs that had not appeared in our search, although had been used in randomized controlled studies in sports research, 15,16 were also selected. In total, 61 condition-specific PROMs were selected for evaluation.…”