Are there roles for observational database studies and structured quantification of expert opinion to answer therapy controversies in transplants?

Abstract

Approaches to determine whether one transplant-related therapy is better than another include: (1) using experimental data, such as those from randomized controlled trials (RCTs); (2) using observational data, such as those from observational databases (ODBs) and (3) using conclusions from the structured quantification of expert opinion based on a consideration of evidence from RCTs, ODBs and other sources. Large RCTs are widely and appropriately regarded as the gold standard of clinical investigation. However, data from large RCTs are rarely available for transplant-related therapy questions. We discuss some of the limitations of RCTs in the transplant setting often including small size and short follow-up. These limitations are only partly solved by meta-analyses of RCTs. Data from high-quality ODBs are not only often useful in this setting but also have limitations. Biases may be difficult or impossible to identify and/or adjust for. However, ODBs have large numbers of diverse subjects receiving diverse therapies and analyses that often give answers more useful to clinicians than RCTs. Side-by-side comparisons suggest analyses from high-quality ODBs often give similar conclusions to meta-analyses of highquality RCTs. Meta-analyses combining data from RCTs and ODBs are sometimes appropriate. Quantitation of expert opinion, when of high quality, is also useful: experts rarely disagree under precisely defined circumstances and their consensus conclusions are often concordant with results of high-quality RCTs and ODBs. We suggest increased use of ODBs and expert opinion as reliable and effective ways to determine relative efficacies of new therapies in transplant settings.

Assessing causal relationships between treatments and clinical outcomes: always read the fine print

“…While the debate about the role of non-randomized studies and comparisons in guiding clinical practice is ongoing, 5,[45][46][47][48][49] empirical evidence seems to suggest that utmost caution is warranted: the list of examples in important public-health settings where the strong expectation of benefit from non-randomized studies has been replaced later by evidence of inefficacy or even harm is unfortunately long and telling (Table 1). In addition to the story of HDCT in breast cancer, it includes hormone replacement therapy for menopausal women (where instead of an expected 40-50% reduction in coronary diseases, 50 randomized data suggested that hormone replacement therapy carries increased risk of coronary disease 51 ), b carotene for lung cancer (where instead of reducing risk of lung cancer as suggested by observational studies (International Agency for Research of Cancer 52 ), randomized data showed increased increase in lung cancer incidence on the b carotene arm 53,54 ), vitamin E for coronary heart disease (where large, well-designed RCTs showed no reduction cardiovascular event rate, 55,56 contrary to the benefit reported in the observational studies 57,58 ) and arthroscopic knee surgery (that was reported to reduce pain in numerous uncontrolled studies 59 and is widely used in the treatment of osteoarthritis yet has not been shown to provide any benefit in randomized trials 59,60 ).…”

Who is the best alternative allotransplant donor?

“…Fifth, randomized trial cannot simulate the diversity of subject-, disease- and transplant-related variable which a physician must consider when recommending a therapy or therapy-strategy. 7 …”

How Helpful Are Meta-Analyses in Determining the Best Therapy of Blood Diseases?

“…We tackled several of the issues in this paper in a previous report [53] with three additional co-authors (M. Eapen, B. Logan and M.J. Zhang) with a far greater knowledge of statistics than ours. The focus of that report was on the value of ODBs in addressing therapy questions in bone marrow and blood cell transplants.…”