Outcome selection, measurement and reporting for new surgical procedures and devices: a systematic review of IDEAL/IDEAL-D studies to inform development of a core outcome set

Abstract

Background: Outcome selection, measurement and reporting for the evaluation of new surgical procedures and devices is inconsistent and lacks standardization. A core outcome set may promote the safe and transparent evaluation of surgical innovations. This systematic review examined outcome selection, measurement and reporting in studies conducted within the IDEAL (Idea, Development, Exploration, Assessment and Long-term monitoring) framework to examine current practice and inform the development of a core outcome set for early-phase studies of surgical procedures/devices. Methods: Web of Science and Scopus citation searches were performed to identify author-reported IDEAL/IDEAL-D studies for any surgical procedure/device. Outcomes were extracted verbatim, including contextual information regarding outcome selection and measurement. Outcomes were categorized to inform a conceptual framework of outcome domains relevant to evaluating innovation. Results: Some 48 studies were identified. Outcome selection, measurement and reporting varied widely across studies in different IDEAL stages. From 1737 outcomes extracted, 22 domains specific to evaluating innovation were conceptualized under seven broad categories: procedure completion success/failure; modifications; unanticipated events; surgeons' experiences; patients' experiences; resource use specific to the innovative procedure/device; and other innovation-specific outcomes. Most innovation-specific outcomes were measured and reported in only a small number of studies. Conclusion: This review highlighted the need for guidance and standardization in outcome selection and reporting in the evaluation of new surgical procedures/devices. Novel outcome domains specific to innovation have been identified to establish a core outcome set for future evaluations of surgical innovations.

Identification of outcomes to inform the development of a core outcome set for surgical innovation: a targeted review of case studies of novel surgical devices

“…By focusing on devices, this review may have failed to identify key innovation outcomes of relevance to the introduction of surgical techniques not involving a device. This review, however, was only one of multiple data sources used to inform the list of outcomes for later COS development 9 25 26…”

Reporting Modifications in Surgical Innovation: A Systematic Scoping Review Protocol

“…This is because modifications are more likely to be discussed in greater detail outside of the existing traditional reporting parameters of surgical research studies and case reports. In addition, review articles and opinion pieces are thought to hold valuable information about desired reporting practices in view of deficient reporting practices in empirical studies [9][10][11].…”

“…The importance of modifications has been recognised in a recent core outcome set developed for early phase surgical studies (the COHESIVE Study), with a 'modifications' domain agreed as one of the core outcome domains to be measured and reported in all studies [8]. Measurement and reporting of modifications in current empirical studies of surgical innovation is, however, heterogenous and lacks detail [9][10][11]. Standardisation of reporting and sharing of modifications is therefore integral to the safe and efficient introduction of new procedures/devices into clinical practice.…”

“…No agreed definition of what constitutes a modification to an evolving surgical procedure currently exists. Likewise, guidance on how modifications should be measured or classified to aid systematic reporting and sharing of modifications is lacking [9,12]. A framework is needed to facilitate transparent reporting and sharing of modifications in future studies of innovative invasive procedures/devices.…”

Using qualitative research methods to understand how surgical procedures and devices are introduced into NHS hospitals: the Lotus study protocol

“…Unlike the pharmaceutical industry, where there are extensive regulatory and ethical requirements,6–8 the introduction of innovative invasive procedures and/or devices (IP/Ds) is not adequately regulated. The Idea, Development, Exploration, Assessment and Long-Term Study (IDEAL) framework9 outlines five consecutive stages of development, evaluation and dissemination of innovative IP/Ds, although its uptake has been slow10 11 and it is often incorrectly followed 12. Furthermore, while innovative IP/Ds should be introduced in the context of formal research studies or in accordance with local hospital policies and processes,13 ethical approval is rarely gained8 and procedures can be introduced without any formal governance from local hospitals 14 15.…”