Efficacy and effectiveness trials have different goals, use different tools, and generate different messages

Abstract

The discussion about the optimal design of clinical trials reflects the perspectives of theory-based scientists and practice-based clinicians. Scientists compare the theory with published results. They observe a continuum from explanatory to pragmatic trials. Clinicians compare the problem they want to solve by completing a clinical trial with the results they can read in the literature. They observe a mixture of what they want and what they get. None of them can solve the problem without the support of the other. Here, we summarize the results of discussions with scientists and clinicians. All participants were interested to understand and analyze the arguments of the other side. As a result of this process, we conclude that scientists tell what they see, a continuum from clear explanatory to clear pragmatic trials. Clinicians tell what they want to see, a clear explanatory trial to describe the expected effects under ideal study conditions and a clear pragmatic trial to describe the observed effects under real-world conditions. Following this discussion, the solution was not too difficult. When we accept what we see, we will not get what we want. If we discuss a necessary change of management, we will end up with the conclusion that two types of studies are necessary to demonstrate efficacy and effectiveness. Efficacy can be demonstrated in an explanatory, ie, a randomized controlled trial (RCT) completed under ideal study conditions. Effectiveness can be demonstrated in an observational, ie, a pragmatic controlled trial (PCT) completed under real-world conditions. It is impossible to design a trial which can detect efficacy and effectiveness simultaneously. The RCTs describe what we may expect in health care, while the PCTs describe what we really observe.

Assessing the efficacy‐effectiveness gap for cancer therapies: A comparison of overall survival and toxicity between clinical trial and population‐based, real‐world data for contemporary parenteral cancer therapeutics

“…In addition, oncologists use survival and toxicity data from clinical trials to counsel patients, inform decision making, and obtain consent for treatment. Alternatively, effectiveness refers to the treatment's performance in the “real‐world” setting, the measurement of which is obtained through observational studies that have high external validity and thus, generate evidence for the generalizability of the therapy's effect . Furthermore, effectiveness research assesses patient, provider, and systemic factors that may influence the intervention's effect and as a result, have an impact on health care policy …”

“…1,2 Clinical trials play a crucial role in informing care for patients with cancer as their results shape the regulatory approval of new treatments and lead to the creation of treatment guidelines. 1,3 In addition, oncologists use survival and toxicity data from clinical trials to counsel patients, inform decision making, and obtain consent for treatment. Alternatively, effectiveness refers to the treatment's performance in the "real-world" setting, the measurement of which is obtained through observational studies that have high external validity and thus, generate evidence for the generalizability of the therapy's effect.…”

“…Alternatively, effectiveness refers to the treatment's performance in the "real-world" setting, the measurement of which is obtained through observational studies that have high external validity and thus, generate evidence for the generalizability of the therapy's effect. 1,4 Furthermore, effectiveness research assesses patient, provider, and systemic factors that may influence the intervention's effect and as a result, have an impact on health care policy. 4,5 An efficacy-effectiveness gap is the difference in patient outcomes as observed in clinical trial data and real-world evidence (RWE).…”

Is there a standard for surgical therapy of hepatocellular carcinoma in healthy and cirrhotic liver? A comparison of eight guidelines

“…Traditional guidelines are based on efficacy but not yet effectiveness data. Methods that compare effectiveness under real-world conditions have only recently been proposed 55. Some of these methods include risk stratification which means that only patients with similar risks (high, low or intermediate) can be compared and the baseline risks of each patient have to be related to each of the outcomes that will be assessed.…”

The Concept of Clinical Economics and its Relation with Effectiveness

“…3 To understand the contribution of PCTs to the existing RCTs we list the differences of these two trials:…”