Review of quality assessment tools for the evaluation of pharmacoepidemiological safety studies

Abstract

ObjectivesPharmacoepidemiological studies are an important hypothesis-testing tool in the evaluation of postmarketing drug safety. Despite the potential to produce robust value-added data, interpretation of findings can be hindered due to well-recognised methodological limitations of these studies. Therefore, assessment of their quality is essential to evaluating their credibility. The objective of this review was to evaluate the suitability and relevance of available tools for the assessment of pharmacoepidemiological safety studies.DesignWe created an a priori assessment framework consisting of reporting elements (REs) and quality assessment attributes (QAAs). A comprehensive literature search identified distinct assessment tools and the prespecified elements and attributes were evaluated.Primary and secondary outcome measuresThe primary outcome measure was the percentage representation of each domain, RE and QAA for the quality assessment tools.ResultsA total of 61 tools were reviewed. Most tools were not designed to evaluate pharmacoepidemiological safety studies. More than 50% of the reviewed tools considered REs under the research aims, analytical approach, outcome definition and ascertainment, study population and exposure definition and ascertainment domains. REs under the discussion and interpretation, results and study team domains were considered in less than 40% of the tools. Except for the data source domain, quality attributes were considered in less than 50% of the tools.ConclusionsMany tools failed to include critical assessment elements relevant to observational pharmacoepidemiological safety studies and did not distinguish between REs and QAAs. Further, there is a lack of considerations on the relative weights of different domains and elements. The development of a quality assessment tool would facilitate consistent, objective and evidence-based assessments of pharmacoepidemiological safety studies.

American Geriatrics Society 2019 Updated AGS Beers Criteria® for Potentially Inappropriate Medication Use in Older Adults

“…Evidence for the benefits and harms of medications in older adults is often limited, particularly from randomized clinical trials, and so decisions on the composition of the criteria were often made in context of best‐available, rather than definitive, evidence. Moreover, evidence assessment frameworks are not perfectly tuned to drug safety evaluation, particularly for observational studies from which much of the relevant evidence derives . The criteria are unable to account for the complexity of all individuals and patient subpopulations, and thus should be taken as guidance to support clinical decision making and not as “the final word” as to whether a specific drug is appropriate or inappropriate for an individual patient.…”

Anticholinergic medication use and dementia: latest evidence and clinical implications

“…We also did not perform a formal evaluation of the quality of the four observational studies using a standardized approach. It is important to note that there is currently no gold standard for evaluating the quality of pharmacoepidemiological studies of drug safety [Neyarapally et al 2012].…”

The Future of Population-Based Postmarket Drug Risk Assessment: A Regulator’s Perspective

“…Further challenges might manifest here if publications of some observational studies chose to be “silent” on the safety aspects of a drug due to space limitations in journals or other considerations. Given all these issues, an effort to develop a framework to evaluate the quality of observational studies in a transparent, consistent way would be of immense value to the field of B‐R or CE evaluation 9 …”

“…When evaluating the quality of evidence from observational pharmacoepidemiologic safety studies, it is necessary to consider relevant aspects including the appropriateness of the patient population; ascertainment of the drug exposure; the accuracy of outcome ascertainment, including the validity of the code or algorithm to identify the health outcome of interest when electronic health data sources are used; and the appropriateness of study design and data analysis, including adequate capture and handling of potential confounders. 9,10 Assessing the quality of spontaneous reports involves, among other factors, the consideration of the description of adverse events and documentation of the diagnosis of the events and information about response to rechallenge and dechallenge. 11 The evaluation of meta-analyses of randomized controlled trials (RCTs) for drug risk assessment involves assessment of randomization and blinding, exposure ascertainment, selection of trials and their suitability for integration, and statistical meta-analytic issues, among many other aspects.…”

“…Given all these issues, an effort to develop a framework to evaluate the quality of observational studies in a transparent, consistent way would be of immense value to the field of B-R or CE evaluation. 9 Furthermore, recent developments in drug safety might further significantly impact the ability to capture effectiveness information in the postmarket phase. By definition, a safety signal is an uncertain finding; the comprehensive and detailed evaluation of the signal is what guides regulatory recommendations, not the existence of the signal itself.…”