Beyond the RCT: When are Randomized Trials Unnecessary for New Therapeutic Devices, and What Should We Do Instead?

Abstract

The aim of this study was to develop an evidence-based framework for evaluation of therapeutic devices, based on ethical principles and clinical evidence considerations Summary Background Data: Nearly all medical products which do not work solely through chemical action are regulated as medical devices. Their huge range of purposes, mechanisms of action and risks pose challenges for regulation. High-profile implantable device failures have fuelled concerns about the level of clinical evidence needed for market approval. Calls for more rigorous evaluation lack clarity about what kind of evaluation is appropriate, and are commonly interpreted as meaning more randomized controlled trials (RCTs). These are valuable where devices are genuinely new and claim to offer measurable therapeutic benefits. Where this is not the case, RCTs may be inappropriate and wasteful. Methods: Starting with a set of ethical principles and basic precepts of clinical epidemiology, we developed a sequential decision-making algorithm for identifying when an RCT should be performed to evaluate new therapeutic devices, and when other methods, such as observational study designs and registry-based approaches, are acceptable. Results: The algorithm clearly defines a group of devices where an RCT is deemed necessary, and the associated framework indicates that an IDEAL 2b study should be the default clinical evaluation method where it is not. Conclusions: The algorithm and recommendations are based on the principles of the IDEAL-D framework for medical device evaluation and appear eminently practicable. Their use would create a safer system for monitoring innovation, and facilitate more rapid detection of potential hazards to patients and the public.

Time to Stop the Witch-Hunt Against Observational Studies

“…Several studies, including systematic reviews and meta-analyses, have shown that the results of well-designed observational studies are reliable and comparable with those of RCTs [16][17][18][19][20][21]. This has prompted the increasing acceptance of well-designed observational studies where RCTs are not practical and earlier results are needed [22,23]. 4.…”

Randomized Clinical Trials of Artificial Intelligence in Medicine: Why, When, and How?

“…However, although methodologically ideal, RCTs may not always be practical for generating real-world evidence about medical devices [ 16 ]. RCTs are time- and resource-intensive.…”

Towards early and broad evaluation of innovative surgical devices: integrating evidence synthesis, stakeholder involvement, and health economic modeling into the clinical research stages of the IDEAL framework

“…9 10 The collaboration aims for evaluation that results in rigorous, safe, and fast evidence gathering, using not only clinical studies but also approaches such as stakeholder consultation, modeling, and cost-effectiveness studies. [11][12][13] The recent attention for these other methodological approaches is important, as clinical trials are expensive and the capacity for conducting these trials is limited. Ideally, only devices that have the greatest potential to improve healthcare and are aligned with the needs and beliefs of involved stakeholders are selected for clinical trials.…”

“…To improve the quality of research for surgical devices, the IDEAL collaboration has adapted their five-stage evaluation framework (Idea, Development, Exploration, Assessment, and Long-term follow-up) to specifically provide recommendations for clinical studies on innovative devices (IDEAL-D) 9 10. The collaboration aims for evaluation that results in rigorous, safe, and fast evidence gathering, using not only clinical studies but also approaches such as stakeholder consultation, modeling, and cost-effectiveness studies 11–13…”