Perceived barriers and facilitators to Risk Based Monitoring in academic-led clinical trials: a mixed methods study

Hurley, Caroline; Sinnott, Carol; Clarke, Mike; Kearney, Patricia M.; Racine, Emmy; Eustace, Joseph A.; Shiely, Frances

doi:10.1186/s13063-017-2148-4

Cited by 21 publications

(23 citation statements)

References 24 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There remains a potential benefit in designing triggers based on existing organisational procedures and checks, which are a result of experience and expertise. Hurley et al identified a lack of knowledge on how to define risks and translate them into monitoring activity as one of the main barriers for trialists to implementing RBM [8]; translating current practice into triggers could be a first step.…”

Section: Discussionmentioning

confidence: 99%

“…The approach is also encouraged by regulators; the International Conference of Harmonisation (ICH) Good Clinical Practice (GCP) guidance (E6[R2]) advises trialists to “develop a systematic, prioritised, risk-based approach to monitoring clinical trials” [6]. RBM tools support one or both of two components of RBM: an initial risk assessment, which determines the overarching monitoring strategy, and support for ongoing monitoring activities in response to the risks identified [7, 8], including determining the nature and frequency of on-site monitoring visits.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Implementing monitoring triggers and matching of triggered and control sites in the TEMPER study: a description and evaluation of a triggered monitoring management system

Díaz-Montaña

Cragg

Choudhury

et al. 2019

Trials

View full text Add to dashboard Cite

Background Triggered monitoring in clinical trials is a risk-based monitoring approach where triggers (centrally monitored, predefined key risk and performance indicators) drive the extent, timing, and frequency of monitoring visits. The TEMPER study used a prospective, matched-pair design to evaluate the use of a triggered monitoring strategy, comparing findings from triggered monitoring visits with those from matched control sites. To facilitate this study, we developed a bespoke risk-based monitoring system: the TEMPER Management System. Methods The TEMPER Management System comprises a web application (the front end), an SQL server database (the back end) to store the data generated for TEMPER, and a reporting function to aid users in study processes such as the selection of triggered sites. Triggers based on current practice were specified for three clinical trials and were implemented in the system. Trigger data were generated in the system using data extracted from the trial databases to inform the selection of triggered sites to visit. Matching of the chosen triggered sites with untriggered control sites was also performed in the system, while data entry screens facilitated the collection and management of the data from findings gathered at monitoring visits. Results There were 38 triggers specified for the participating trials. Using these, 42 triggered sites were chosen and matched with control sites. Monitoring visits were carried out to all sites, and visit findings were entered into the TEMPER Management System. Finally, data extracted from the system were used for analysis. Conclusions The TEMPER Management System made possible the completion of the TEMPER study. It implemented an approach of standardising the automation of current-practice triggers, and the generation of trigger data to inform the selection of triggered sites to visit. It also implemented a matching algorithm informing the selection of matched control sites. We hope that by publishing this paper it encourages other trialists to share their approaches to, and experiences of, triggered monitoring and other risk-based monitoring systems.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Implementing monitoring triggers and matching of triggered and control sites in the TEMPER study: a description and evaluation of a triggered monitoring management system

Díaz-Montaña

Cragg

Choudhury

et al. 2019

Trials

View full text Add to dashboard Cite

show abstract

“…As well as uncertainty about how to develop relevant methods, uncertainty or concern about costs involved is a substantial barrier to adoption of risk-based monitoring. 53 Through our various searches, we identified 24 commercial companies which, we were at least reasonably certain, had a method relevant to our search (data not shown). Scoping review methodology advises that relevant experts in a field are surveyed to help identify other relevant work.…”

Section: /47mentioning

confidence: 99%

Dynamic methods for ongoing assessment of site-level risk in risk-based monitoring of clinical trials: a scoping review

Cragg

Hurley

Yorke-Edwards

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Background/Aims It is increasingly recognised that reliance on frequent site visits for monitoring clinical trials is inefficient. Regulators and trialists have in recent years encouraged more risk-based monitoring. Risk assessment should take place before a trial begins in order to define the overarching monitoring strategy. It can also be done on an ongoing basis, in order to target sites for monitoring activity. Various methods have been proposed for such prioritisation, often using terms like 'central statistical monitoring', 'triggered monitoring' or, as in ICH Good Clinical Practice guidance, 'targeted on-site monitoring'. We conducted a scoping review to identify such methods, to establish if any published methods were supported by adequate evidence to allow wider implementation, and to point the way to future developments in this field of research. Methods We used 7 publication databases, 2 sets of methodological conference abstracts and an internet search engine to look for methods for using centrally held trial data to assess site conduct during a trial. We included only reports in English, and excluded reports published before 1996 and reports not directly relevant to our research question. We used reference and citation searches to find additional relevant reports. We extracted data using a pre-defined template. We contacted authors to request additional information about included reports and to check whether reports might be eligible. Results We included 30 reports in our final dataset, of which 21 were peer-reviewed publications. 20 reports described central statistical monitoring methods (of which 7 focussed on detection of fraud or misconduct) and 9 described triggered monitoring methods. 21 reports included some assessment of their methods' effectiveness. Most commonly this involved exploring the methods' characteristics using real trial data with no known integrity issues. Of the 21 with some effectiveness assessment, most presented limited or no information about whether or not concerns identified through central monitoring constituted meaningful problems. Some reports commented on cost savings from reduced on-site monitoring, but none gave detailed costings for the development and maintenance of central monitoring methods themselves. Conclusions Our review identified various proposed methods, some of which could be combined within the same trial. The apparent emphasis on fraud detection may not be proportionate in all trial settings. Although some methods have self-justifying benefits for data cleaning activity, many have limitations that may currently prevent their routine use for targeting trial monitoring activity. The implementation costs, or uncertainty about these, may also be a barrier. We make recommendations for how the evidence-base supporting these methods could be improved. Keywords: trial monitoring, risk-based monitoring, triggered monitoring, central statistical monitoring, Good Clinical Practice, research misconduct, data fabrication

show abstract

“…The increasing number of clinical trials and sites and the growing complexity of study designs make it challenging to detect and identify clinical quality issues timely. Traditional site monitoring strategies, which rely on on-site visits with source data verification (SDV) and on risk-based approaches, are also attempting to mitigate the risk of occurrence of clinical quality issues [3,4]. However, despite extensive audit programs and monitoring activities performed by sponsors, issues related to data integrity, safety, sponsor oversight, protection of primary endpoints and patient consent are recurring audit and inspection findings [5,6].…”

Section: Introductionmentioning

confidence: 99%

Harnessing the Power of Quality Assurance Data: Can We Use Statistical Modeling for Quality Risk Assessment of Clinical Trials?

Koneswarakantha¹,

Ménard²,

Rolo

et al. 2020

Ther Innov Regul Sci

View full text Add to dashboard Cite

Background The increasing number of clinical trials and their complexity make it challenging to detect and identify clinical quality issues timely. Despite extensive sponsor audit programs and monitoring activities, issues related to data integrity, safety, sponsor oversight and patient consent have recurring audit and inspection findings. Recent developments in data management and IT systems allow statistical modeling to provide insights to clinical Quality Assurance (QA) professionals to help mitigate some of the key clinical quality issues more holistically and efficiently. Methods We used findings from a curated data set from Roche/Genentech operational and quality assurance study data, covering a span of 8 years (2011-2018) and grouped them into 5 clinical impact factor categories, for which we modeled the risk with a logistic regression using hand crafted features. Results We were able to train 5 interpretable, cross-validated models with several distinguished risk factors, many of which confirmed field observations of our quality professionals. Our models were able to reliably predict a decrease in risk by 12-44%, with 2-8 coefficients each, despite a low signal-to-noise ratio in our data set. Conclusion We proposed a modeling strategy that could provide insights to clinical QA professionals to help them mitigate key clinical quality issues (e.g., safety, consent, data integrity) in a more sustained data-driven way, thus turning the traditional reactive approach to a more proactive monitoring and alerting approach. Also, we are calling for cross-sponsors collaborations and data sharing to improve and further validate the use of statistical models in clinical QA.

show abstract

Perceived barriers and facilitators to Risk Based Monitoring in academic-led clinical trials: a mixed methods study

Cited by 21 publications

References 24 publications

Implementing monitoring triggers and matching of triggered and control sites in the TEMPER study: a description and evaluation of a triggered monitoring management system

Implementing monitoring triggers and matching of triggered and control sites in the TEMPER study: a description and evaluation of a triggered monitoring management system

Dynamic methods for ongoing assessment of site-level risk in risk-based monitoring of clinical trials: a scoping review

Harnessing the Power of Quality Assurance Data: Can We Use Statistical Modeling for Quality Risk Assessment of Clinical Trials?

Contact Info

Product

Resources

About