Enhancing the Credibility of Decisions Based on Scientific Conclusions: Transparency Is Imperative

Abstract

Transparency and documentation of the decision process are at the core of a credible risk assessment and, in addition, are essential in the presentation of a weight of evidence (WoE)-based approach. Lack of confidence in the risk assessment process (as the basis for a risk management decision), beginning with evaluation of raw data and continuing through the risk decision process, is largely because of issues surrounding transparency. There is a critical need to implement greater transparency throughout the risk assessment process, and although doing so will not guarantee the correctness of the risk assessment or that all risk assessors come up with the same conclusions, it will provide essential information on how a particular conclusion or decision was made, thereby increasing confidence in the conclusions. Recognizing this issue, the International Life Sciences Institute Health and Environmental Sciences Institute convened a multisector committee tasked with discussing this issue and examining existing guidance and recommendations related to transparency in risk assessment. The committee concluded that transparency is inextricably linked to credibility: credibility of the data, credibility of the risk assessment process, and credibility of the resulting decision making. To increase this credibility, existing guidance concerning criteria elements of transparency related to the risk assessment process must be more widely disseminated and applied, and raw data for studies used in human health and environmental risk assessment must be more widely available. Finally, the decision-making process in risk management must be better documented and a guidance framework established for both the process itself and its communication to the public.

Problems with Traditional Science Publishing and Finding a Wider Niche for Post-Publication Peer Review

“…Then there is the issue of data, the fundamental basis of science, and the information based on which the public extends its credibility in the scientific community and what is achieved by it. Unless data that is reported in scientific publications is valid, credible, and reproducible, and unless there is a transparent system of evaluation in place to assess that data (Schreider et al, 2010), the principle of good laboratory practice (GLP) standards employed for compliance with regulatory mandates (McCarty et al, 2012) will become redundant and useless, and the efforts made by some to comply with GLP standards will be dwarfed by those that do not. The lack of reproducibility of data is a serious problem, especially when scientists are financially rewarded (and not penalized) for papers that carry fake, unreliable, or unreproducible data sets (Hartshorne and Schachtner, 2012).…”

“…Duplicate publications distort the metrics of journals and of the literature (Norman and Griffiths, 2008), provide an unfair advantage to the scientist that has duplicated the data, and serve to erode publishing ethics if allowed to continue unpunished or uncorrected. Chambers (2013) lists the major problems associated with publication in the life sciences as: bias (Thornton and Lee, 2000;Fanelli, 2010), insufficient statistical power (Tressoldi, 2012), poor replicability (Pashler and Wagenmakers, 2012), undisclosed analytic flexibility (Wagenmakers, 2007;Bakker et al, 2012;Masicampo and Lalande, 2012), and a lack of data transparency (Ioannidis, 2005;Schreider et al, 2010). John et al (2012) show that 67% of researchers in a survey of over 2,000 psychologists admitted (anonymously) to selectively reporting experiments that produce desirable outcomes, 72% a P value fishing, or 78% failing to report all dependent variables in an experiment, which could incentivize "a host of questionable practices that twist the evidence to suit the narrative" (Chambers, 2013).…”

“…This can be guaranteed by using strict quality control (QC), which lies in the hands of another triade, the author, the editor (including the editor-in-chief [EIC]), and the publisher (Teixeira da Silva, 2013a; Teixeira da . The current (traditional) system of scientific publishing uses a traditional peer review (TPR) to evaluate papers before acceptance and publication that often lacks quality and transparency (Schreider et al, 2010;Balhara, 2012). Problems, listed in Table 1, that may weaken the effectiveness of traditional peer review (TPR) and thus result in publication of scientifically low-level papers can be classified into four main groups: i) lack of quality which may reflect a paper's lack of quality, the lack of quality control (QC) by Downloaded by [Ryerson University] at 06:55 03 December 2014 reviewers, or the lack or low level of peer-review due to a limited or unavailable peer pool; ii) lack of transparency (Schreider et al, 2010) or no transparency during TPR; (iii) bias; and (iv) fraud.…”

Evidence-based toxicology for the 21st century: Opportunities and challenges

“…the launching of the eBtC is timely, as the toxicology literature increasingly invokes eB-related themes or practices such as transparency in decision-making (Schreider et al, 2010), systematic and transparent reviews of evidence , synthesis of types of evidence to establish causal inference (Adami et al, 2011), and assessment of bias/credibility (Conrad and Becker, 2010). We also see practical examples of the application of evidence-based methodology or terminology Abhyankar et al, 2011;Maull et al, 2012).…”

Integrated Testing Strategy (ITS) – Opportunities to better use existing data and guide future testing in toxicology

“…We acknowledge that it is a useful tool for today's chemical hazard and risk assessment. However, we doubt its use regarding ItS, primarily because it lacks a methodological basis for making transparent consistent inferences, as highlighted by Jaworska et al (2010a) and Schreider et al (2010). Schreider et al (2010) mainly link credibility to transparency, but transparency is also a crucial prerequisite to improving the risk assessment process regarding the methodology used.…”