When and how to use data from randomised trials to develop or validate prognostic models

Pajouheshnia, Romin; Groenwold, Rolf H H; Peelen, Linda M.; Reitsma, Johannes B.; Moons, Karel G.M.

doi:10.1136/bmj.l2154

Cited by 27 publications

(16 citation statements)

References 62 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another challenge with the AI‐based methods is often the need for substantial amounts of high‐quality data to improve the performance of the model. Interestingly, 25 000 RCTs of treatment interventions were published in 2018, and RCT data can be considered the gold standard in most settings 52 . These RCT data can seamlessly be reused to generate AI‐based models and collaborate to combine the patient‐level data from several RCTs.…”

Section: Discussionmentioning

confidence: 99%

“…Interestingly, 25 000 RCTs of treatment interventions were published in 2018, and RCT data can be considered the gold standard in most settings. 52 These RCT data can seamlessly be reused to generate AI-based models and collaborate to combine the patient-level data from several RCTs. The randomized nature of RCTs additionally provides essential elements in the development of prognostic models by replicating real-life perspective setting.…”

Section: Artificial Intelligence Applications In Biliary Disordersmentioning

confidence: 99%

See 1 more Smart Citation

Artificial intelligence in pancreaticobiliary endoscopy

Akshintala

Khashab

2021

J of Gastro and Hepatol

View full text Add to dashboard Cite

Artificial intelligence (AI) applications in health care have exponentially increased in recent years, and a few of these are related to pancreatobiliary disorders. AI‐based methods were applied to extract information, in prognostication, to guide clinical treatment decisions and in pancreatobiliary endoscopy to characterize lesions. AI applications in endoscopy are expected to reduce inter‐operator variability, improve the accuracy of diagnosis, and assist in therapeutic decision‐making in real time. AI‐based literature must however be interpreted with caution given the limited external validation. A multidisciplinary approach combining clinical and imaging or endoscopy data will better utilize AI‐based technologies to further improve patient care.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Artificial Intelligence Applications In Biliary Disordersmentioning

confidence: 99%

Artificial intelligence in pancreaticobiliary endoscopy

Akshintala

Khashab

2021

J of Gastro and Hepatol

View full text Add to dashboard Cite

show abstract

“…The use of clinical trial data is potentially challenging, particularly if there is a large treatment effect observed in the trial. The potential of using clinical trial data has recently been described by Pajouheshnia et al 120 Options are to include treatment (if the effect is large) in the model or to use the control arm only (particularly if the arm was a placebo arm). The KAT observed no differences in OKS between the treatment arms, which reinforced our justification of using the entire trial data.…”

Section: Discussionmentioning

confidence: 99%

Outpatient physiotherapy versus home-based rehabilitation for patients at risk of poor outcomes after knee arthroplasty: CORKA RCT

Barker

Room

Knight

et al. 2020

Health Technol Assess

View full text Add to dashboard Cite

Background Over 100,000 primary knee arthroplasty operations are undertaken annually in the UK. Around 15–30% of patients do not report a good outcome. Better rehabilitation strategies may improve patient-reported outcomes. Objectives To compare the outcomes from a traditional outpatient physiotherapy model with those from a home-based rehabilitation programme for people assessed as being at risk of a poor outcome after knee arthroplasty. Design An individually randomised, two-arm controlled trial with a blinded outcome assessment, a parallel health economic evaluation and a nested qualitative study. Setting The trial took place in 14 NHS physiotherapy departments. Participants People identified as being at high risk of a poor outcome after knee arthroplasty. Interventions A multicomponent home-based rehabilitation package delivered by rehabilitation assistants with supervision from qualified therapists compared with usual-care outpatient physiotherapy. Main outcome measures The primary outcome was the Late Life Function and Disability Instrument at 12 months. Secondary outcomes were the Oxford Knee Score (a disease-specific measure of function); Knee injury and Osteoarthritis Outcome Score; Quality of Life subscale; Physical Activity Scale for the Elderly; EuroQol-5 Dimensions, five-level version; and physical function assessed using the Figure-of-8 Walk Test, 30-Second Chair Stand Test and Single Leg Stance. Data on the use of health-care services, time off work and informal care were collected using participant diaries. Results In total, 621 participants were randomised. A total of 309 participants were assigned to the COmmunity based Rehabilitation after Knee Arthroplasty (CORKA) home-based rehabilitation programme, receiving a median of five treatment sessions (interquartile range 4–7 sessions). A total of 312 participants were assigned to usual care, receiving a median of four sessions (interquartile range 2–6 sessions). The primary outcome, Late Life Function and Disability Instrument function total score at 12 months, was collected for 279 participants (89%) in the home-based CORKA group and 287 participants (92%) in the usual-care group. No clinically or statistically significant difference was found between the groups (intention-to-treat adjusted difference 0.49 points, 95% confidence interval –0.89 to 1.88 points; p = 0.48). There were no statistically significant differences between the groups in any of the patient-reported or physical secondary outcome measures at 6 or 12 months post randomisation. The health economic analysis found that the CORKA intervention was cheaper to provide than usual care (£66 less per participant). Total societal costs (combining health-care costs and other costs) were lower for the CORKA intervention than usual care (£316 less per participant). Adopting a societal perspective, CORKA had a 75% probability of being cost-effective at a threshold of £30,000 per quality-adjusted life-year. Adopting the narrower health and social care perspective, CORKA had a 43% probability of being cost-effective at the same threshold. Limitations The interventions were of short duration and were set within current commissioning guidance for UK physiotherapy. Participants and treating therapists could not be blinded. Conclusions This randomised controlled trial found no important differences in outcomes when post-arthroplasty rehabilitation was delivered using a home-based, rehabilitation assistant-delivered rehabilitation package or a traditional outpatient model. However, the health economic evaluation found that when adopting a societal perspective, the CORKA home-based intervention was cost-saving and more effective than, and thus dominant over, usual care, owing to reduced time away from paid employment for this group. Further research could look at identifying the risk of poor outcome and further evaluation of a cost-effective treatment, including the workforce model to deliver it. Trial registration Current Controlled Trials ISRCTN13517704. Funding This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 65. See the NIHR Journals Library website for further project information.

show abstract

“…However, a small sample size is common in prediction studies that use data from RCTs and would have been less of an issue if the chosen predictors had been stronger. 48 Second, adding more than one predictor-treatment interaction caused the models to become unstable. We therefore only added the interaction with the lowest p value to the model.…”

Section: Discussionmentioning

confidence: 99%

Treatment success of internet-based vestibular rehabilitation in general practice: development and internal validation of a prediction model

et al. 2020

View full text Add to dashboard Cite

ObjectivesTo develop and internally validate prediction models to assess treatment success of both stand-alone and blended online vestibular rehabilitation (VR) in patients with chronic vestibular syndrome.DesignSecondary analysis of a randomised controlled trial.Setting59 general practices in The Netherlands.Participants202 adults, aged 50 years and older with a chronic vestibular syndrome who received either stand-alone VR (98) or blended VR (104). Stand-alone VR consisted of a 6-week, internet-based intervention with weekly online sessions and daily exercises. In blended VR, the same intervention was supplemented with physiotherapy support.Main outcome measuresSuccessful treatment was defined as: clinically relevant improvement of (1) vestibular symptoms (≥3 points improvement Vertigo Symptom Scale—Short Form); (2) vestibular-related disability (>11 points improvement Dizziness Handicap Inventory); and (3) both vestibular symptoms and vestibular-related disability. We assessed performance of the predictive models by applying calibration plots, Hosmer-Lemeshow statistics, area under the receiver operating characteristic curves (AUC) and applied internal validation.ResultsImprovement of vestibular symptoms, vestibular-related disability or both was seen in 121, 81 and 64 participants, respectively. We generated predictive models for each outcome, resulting in different predictors in the final models. Calibration for all models was adequate with non-significant Hosmer-Lemeshow statistics, but the discriminative ability of the final predictive models was poor (AUC 0.54 to 0.61). None of the identified models are therefore suitable for use in daily general practice to predict treatment success of online VR.ConclusionIt is difficult to predict treatment success of internet-based VR and it remains unclear who should be treated with stand-alone VR or blended VR. Because we were unable to develop a useful prediction model, the decision to offer stand-alone or blended VR should for now be based on availability, cost effectiveness and patient preference.Trial registration numberThe Netherlands Trial Register NTR5712.

show abstract

When and how to use data from randomised trials to develop or validate prognostic models

Cited by 27 publications

References 62 publications

Artificial intelligence in pancreaticobiliary endoscopy

Artificial intelligence in pancreaticobiliary endoscopy

Outpatient physiotherapy versus home-based rehabilitation for patients at risk of poor outcomes after knee arthroplasty: CORKA RCT

Treatment success of internet-based vestibular rehabilitation in general practice: development and internal validation of a prediction model

Contact Info

Product

Resources

About