No abstract

The promise of artificial intelligence (AI) in health care offers substantial opportunities to improve patient and clinical team outcomes, reduce costs, and influence population health. Current data generation greatly exceeds human cognitive capacity to effectively manage information, and AI is likely to have an important and complementary role to human cognition to support delivery of personalized health care. 1 For example, recent innovations in AI have shown high levels of accuracy in imaging and signal detection tasks and are considered among the most mature tools in this domain. 2 However, there are challenges in realizing the potential for AI in health care. Disconnects between reality and expectations have led to prior precipitous declines in use of the technology, termed AI winters, and another such event is possible, especially in health care. 3 Today, AI has outsized market expectations and technology sector investments. Current challenges include using biased data for AI model development, applying AI outside of populations represented in the training and validation data sets, disregarding the effects of possible unintended consequences on care or the patientclinician relationship, and limited data about actual effects on patient outcomes and cost of care.AI in Healthcare: The Hope, The Hype, The Promise, The Peril, a publication by the National Academy of Medicine (NAM), synthesizes current knowledge and offers a reference document for the responsible development, implementation, and maintenance of AI in the clinical enterprise. 4 The publication outlines current and near-term AI solutions; highlights the challenges, limi-Health care is at a critical juncture for the safe and effective use of AI algorithms and tools in supporting the health of patients.

There is a growing appreciation that our current approach to clinical research leaves important gaps in evidence from the perspective of patients, clinicians, and payers wishing to make evidence-based clinical and health policy decisions. This has been a major driver in the rapid increase in interest in comparative effectiveness research (CER), which aims to compare the benefits, risks, and sometimes costs of alternative health-care interventions in 'the real world'. While a broad range of experimental and nonexperimental methods will be used in conducting CER studies, many important questions are likely to require experimental approaches - that is, randomized controlled trials (RCTs). Concerns about the generalizability, feasibility, and cost of RCTs have been frequently articulated in CER method discussions. Pragmatic RCTs (or 'pRCTs') are intended to maintain the internal validity of RCTs while being designed and implemented in ways that would better address the demand for evidence about real-world risks and benefits for informing clinical and health policy decisions. While the level of interest and activity in conducting pRCTs is increasing, many challenges remain for their routine use. This article discusses those challenges and offers some potential ways forward.

We conclude that a CER-ECD observational study that imposes no or minimal additional risk to or burden on patients may proceed ethically without express informed consent from participants in settings where: (a) patients are regularly informed of the health care institution's commitment to learning through the integration of research and practice; and (b) there are appropriate protections for patients' rights and interests. In addition, where (a) and (b) apply, some pragmatic, randomized trials that similarly impose no or minimal additional risk to or burden on patients may also proceed ethically without express consent, when certain additional conditions are satisfied, including: (c) the trial does not negatively affect patients' prospects for good clinical outcomes; (d) physicians have the option of using an intervention other than the one assigned if they believe doing so is important for a particular patient; and (e) the trial does not engage preferences or values that are meaningful to patients.

BackgroundAbout 30 million individuals in the United States are living with a rare disease, which by definition have a prevalence of 200,000 or fewer cases in the United States ([National Organization for Rare Disorders], [About NORD], [2016]). Disease heterogeneity and geographic dispersion add to the difficulty of completing robust studies in small populations. Improving the ability to conduct research on rare diseases would have a significant impact on population health. The purpose of this paper is to raise awareness of methodological approaches that can address the challenges to conducting robust research on rare diseases.ApproachWe conducted a landscape review of available methodological and analytic approaches to address the challenges of rare disease research. Our objectives were to: 1. identify algorithms for matching study design to rare disease attributes and the methodological approaches applicable to these algorithms; 2. draw inferences on how research communities and infrastructure can contribute to the efficiency of research on rare diseases; and 3. to describe methodological approaches in the rare disease portfolio of the Patient-Centered Outcomes Research Institute (PCORI), a funder promoting both rare disease research and research infrastructure.ResultsWe identified three algorithms for matching study design to rare disease or intervention characteristics (Gagne, et.al, BMJ 349:g6802, 2014); (Gupta, et.al, J Clin Epidemiol 64:1085-1094, 2011); (Cornu, et. al, Orphet J Rare Dis 8:48,2012) and summarized the applicable methodological and analytic approaches. From this literature we were also able to draw inferences on how an effective research infrastructure can set an agenda, prioritize studies, accelerate accrual, catalyze patient engagement and terminate poorly performing studies. Of the 24 rare disease projects in the PCORI portfolio, 11 are randomized controlled trials (RCTs) using standard designs. Thirteen are observational studies using case-control, prospective cohort, or natural history designs. PCORI has supported the development of 9 Patient-Powered Research Networks (PPRNs) focused on rare diseases.ConclusionMatching research design to attributes of rare diseases and interventions can facilitate the completion of RCTs that are adequately powered. An effective research infrastructure can improve efficiency and avoid waste in rare disease research. Our review of the PCORI research portfolio demonstrates that it is feasible to conduct RCTs in rare disease. However, most of these studies are using standard RCT designs. This suggests that use of a broader array of methodological approaches to RCTs --such as adaptive trials, cross-over trials, and early escape designs can improve the productivity of robust research in rare diseases.

There is growing emphasis on eliciting and incorporating stakeholder perspectives into health research and public policy development. The deliberative engagement session (DES) method provides one approach to elicit informed preferences from patients and other stakeholders on policy issues. DES involves day-long interaction with participants, including short plenary presentations followed by small group discussion. While interest in DES methods is expanding, practical guidance for researchers on this method remains limited. In this paper, we describe the DES method and its contemporary relevance for health policy research, illustrate how to conduct a DES using an example of a recent patient-centered outcomes research (PCOR) study with which we were involved, and discuss strengths and challenges of using this approach. DES methods generate rich data, reduce the risk of eliciting uniformed preferences or non-attitudes, and increase the likelihood of eliciting informed, reflective preferences. However, they are resource-intensive, and thus generally require trading away a larger, more representative sample. Despite these limitations, the DES method, when carefully designed, is well-suited for engaging stakeholders in research on complex health policy issues.

Background: Informed consent requirements generally require a lengthy process and signed documentation for patients to participate in clinical research. With growing interest in comparative effectiveness research (CER), whereby patients receive approved (nonexperimental) medicines for their medical condition, questions have been raised whether the same consent requirements should apply. Little input from patients has been part of these debates. Methods: We conducted two "deliberative engagement sessions" with patients from Johns Hopkins Community Physicians (JHCP) and Geisinger Health System (GHS). Full-day sessions introduced participants to two different CER designs (observational vs. randomized) comparing two antihypertensive medications and three disclosure or consent approaches: Opt-In, Opt-Out, and "General Approval." Sessions consisted of presentations and extensive discussion at small group tables. Pre-and posttest surveys were completed by participants before and after all-day discussions measuring attitudes about research and about each of the three disclosure/ consent options. Results: One hundred thirty-seven adults over age 40 years participated. Attitudes were similar between JHCP and GHS. Participants strongly preferred Opt-In or Opt-Out consent options to General Approval for both observational and randomized designs. For the randomized CER study, 70% liked Opt-In, 65% liked Opt-Out, and 40% liked General Approval. In discussing disclosure/consent options, patients cared most about choice, information, privacy and confidentiality, quality of the research, trust, respect, and impact of the study on patient care. Conclusions: The majority of participants from two different types of health systems liked both Opt-In and Opt-Out approaches for observational and randomized designs for low-risk CER. There were no posttest differences in the proportion liking Opt-In versus Opt-Out. Patients in this study wanted to be told about research and have a choice, but were very open to such disclosures being streamlined. Policymakers may find patients' views about what matters to them in the context of consent and CER relevant.

To successfully implement a pragmatic clinical trial, investigators need access to numerous resources, including financial support, institutional infrastructure (e.g., clinics, facilities, staff), eligible patients, and patient data. Gatekeepers are people or entities who have the ability to allow or deny access to the resources required to support the conduct of clinical research. Based on this definition, gatekeepers relevant to the United States clinical research enterprise include research sponsors, regulatory agencies, payers, health system and other organizational leadership, research team leadership, human research protections programs, advocacy and community groups, and clinicians. This manuscript provides a framework to help guide gatekeepers’ decision-making related to the use of resources for pragmatic clinical trials. These include (1) concern for the interests of individuals, groups, and communities affected by the gatekeepers’ decisions, including protection from harm and maximization of benefits, (2) advancement of organizational mission and values, and (3) stewardship of financial, human, and other organizational resources. Separate from these ethical considerations, gatekeepers’ actions will be guided by relevant federal, state, and local regulations. This framework also suggests that to further enhance the legitimacy of their decision-making, gatekeepers should adopt transparent processes that engage relevant stakeholders when feasible and appropriate. We apply this framework to the set of gatekeepers responsible for making decisions about resources necessary for pragmatic clinical trials in the United States, describing the relevance of the criteria in different situations and pointing out where conflicts among the criteria and relevant regulations may affect decision-making. Recognition of the complex set of considerations that should inform decision-making will guide gatekeepers in making justifiable choices regarding the use of limited and valuable resources.