Evaluating Quality of Decision-Making Processes in Medicines' Development, Regulatory Review, and Health Technology Assessment: A Systematic Review of the Literature

Abstract

Introduction: Although pharmaceutical companies, regulatory authorities, and health technology assessment (HTA) agencies have been increasingly using decision-making frameworks, it is not certain whether these enable better quality decision making. This could be addressed by formally evaluating the quality of decision-making process within those organizations. The aim of this literature review was to identify current techniques (tools, questionnaires, surveys, and studies) for measuring the quality of the decision-making process across the three stakeholders.Methods: Using MEDLINE, Web of Knowledge, and other Internet-based search engines, a literature review was performed to systematically identify techniques for assessing quality of decision making in medicines development, regulatory review, and HTA. A structured search was applied using key words and a secondary review was carried out. In addition, the measurement properties of each technique were assessed and compared. Ten Quality Decision-Making Practices (QDMPs) developed previously were then used as a framework for the evaluation of techniques identified in the review. Due to the variation in studies identified, meta-analysis was inappropriate.Results: This review identified 13 techniques, where 7 were developed specifically to assess decision making in medicines' development, regulatory review, or HTA; 2 examined corporate decision making, and 4 general decision making. Regarding how closely each technique conformed to the 10 QDMPs, the 13 techniques assessed a median of 6 QDMPs, with a mode of 3 QDMPs. Only 2 techniques evaluated all 10 QDMPs, namely the Organizational IQ and the Quality of Decision Making Orientation Scheme (QoDoS), of which only one technique, QoDoS could be applied to assess decision making of both individuals and organizations, and it possessed generalizability to capture issues relevant to companies as well as regulatory authorities.Conclusion: This review confirmed a general paucity of research in this area, particularly regarding the development and systematic application of techniques for evaluating quality decision making, with no consensus around a gold standard. This review has identified QoDoS as the most promising available technique for assessing decision making in the lifecycle of medicines and the next steps would be to further test its validity, sensitivity, and reliability.

A Process for Evaluating Quality Decision-Making Practices During the Development, Review and Reimbursement of Medicines

“…6 A recent review of the literature identified general paucity of research in this area, particularly regarding the development and systematic application of techniques for evaluating quality decision-making, such as through the studies undertaken as part of this research. 9 Indeed this is the first assessment to evaluate quality of decision-making within the different organisations involved in the development, review and reimbursement of medicines.…”

“…A systematic review of the literature demonstrated QoDoS to be the most promising available technique for such assessments. 9 Moreover, a pilot study with participants from pharmaceutical companies and regulatory authorities demonstrated the initial practicality of the QoDoS in assessing the strengths and weaknesses as well as similarities and differences in decision-making practices amongst individuals as well as their perception of their organisation. 10 Furthermore, a recent study demonstrated the reliability and relevance of QoDoS in the decision-making of pharmaceutical companies, regulatory authorities and HTA agencies.…”

The Reliability and Relevance of a Quality of Decision Making Instrument, Quality of Decision-Making Orientation Scheme (QoDoS), for Use During the Lifecycle of Medicines

“…The practicality of the tool in a regulatory authority and pharmaceutical company setting was confirmed through a study with 76 participants (50% from regulatory authorities and 50% from pharmaceutical companies) (Bujar et al, 2016). The findings of this pilot study as well as the results of a recent systematic review (Bujar et al, 2017) have demonstrated that the QoDoS is considered to be the most promising instrument for evaluating quality of the decision-making process during medicines development and review, identifying strengths and weaknesses and raising awareness of the issues in quality decision making across individuals and within organisations. The challenge is how to ensure that the QoDoS, in addition to the above described attributes, produces reliable, and relevant results.…”

Practical applications of regulatory requirements for signal detection and communications in pharmacovigilance

“…The CIRS-BRAT tool allows users to generate tabular and graphical displays to assist in the interpretation of benefit and risk findings. 33 The purpose of the tool is to enable users to generate value trees, key benefit-risk summary tables, and forest plots. 33 The ICH guideline M4E entitled "Common technical document for the registration of pharmaceuticals for human use -Efficacy" was revised in order to standardize the content and presentation of benefit-risk information in regulatory submissions.…”

“…33 The purpose of the tool is to enable users to generate value trees, key benefit-risk summary tables, and forest plots. 33 The ICH guideline M4E entitled "Common technical document for the registration of pharmaceuticals for human use -Efficacy" was revised in order to standardize the content and presentation of benefit-risk information in regulatory submissions. The Revised ICH M4E(R2) guideline specifies a structure for the Common Technical Document (CTD) in Section 2.5.6, Benefits and Risk Conclusions.…”