Most network meta-analyses include only randomized controlled trials (RCTs), so we 62 will focus on this study design, and on relative treatment effects. A network meta-analysis 63 involves the integration of direct and indirect evidence in a network of relevant trials. We 64 assume that evaluation of the credibility of results takes place once all primary analyses and 65 sensitivity analyses have been undertaken. We assume that reviewers have implemented 66 their pre-specified study inclusion criteria, which may include risk of bias considerations, 67 and have obtained the best possible estimates of relative treatment effects using 68 appropriate statistical methods (e.g. those described in (7-10)). The question is then how to 69 make judgements about the credibility of relative treatment effects, given that trials with 70 variable risk of bias, precision, relevance and heterogeneity contribute information to the 71 estimate. 72This paper addresses how judgements should be formed about the six CINeMA 73 domains. We illustrate the methods using three examples: a network of trials that compare 74 outcomes of various diagnostic strategies in patients with suspected acute coronary 75 syndrome (11), a network of trials comparing the effectiveness of 18 antidepressants for 76 major depression (12), and a network comparing adverse events of statins (13). The three 77 examples are introduced in Error! Reference source not found.. All analyses were done in R 78 software using the netmeta package and the CINeMA web application (Box 2) (6,14). 79
WITHIN-STUDY BIAS
80
BACKGROUND AND DEFINITIONS
81Within-study bias refers to shortcomings in the design or conduct of a study that can 82 lead to an estimated relative treatment effect that systematically differs from the truth. In 83 5 our framework we assume that studies have been assessed for risk of bias. The majority of 84 published systematic reviews of RCTs currently use a tool developed by Cochrane to 85 evaluate risk of bias (15). This tool classifies studies as having low, unclear or high risk of 86 bias for various bias components (such as allocation concealment, attrition, blinding etc.), 87 and these judgements are then summarized across domains. A revision of the tool takes a 88 similar approach but labels the levels as low risk of bias, some concerns and high risk of bias 89 (16). 90
THE CINEMA APPROACH
91While it is straightforward to gauge the impact of within-study biases on the summary 92 relative treatment effect in a pairwise meta-analysis (17), in network meta-analysis studies 93 contribute data to the estimation of each summary effect in a complex manner. In the first 94 example discussed below we show the complexity underpinning the flow of information in 95 the network of diagnostic modalities used to detect coronary artery disease. A treatment 96 comparison directly evaluated in studies with low risk of bias might also be estimated 97 indirectly (via a common comparator) using studies at high risk of bias, and vice versa. While 98 studies at low risk of ...