Bias in Epidemiological Studies of Conflict Mortality

Abstract

Cluster sampling has recently been used to estimate the mortality in various conflicts around the world. The Burnham et al. study on Iraq employs a new variant of this cluster sampling methodology. The stated methodology of Burnham et al. is to (1) select a random main street, (2) choose a random cross street to this main street, and (3) select a random household on the cross street to start the process. The authors show that this new variant of the cluster sampling methodology can introduce an unexpected, yet substantial, bias into the resulting estimates, as such streets are a natural habitat for patrols, convoys, police stations, roadblocks, cafes, and street-markets. This bias comes about because the residents of households on cross-streets to the main streets are more likely to be exposed to violence than those living further away. Here, the authors develop a mathematical model to gauge the size of the bias and use the existing evidence to propose values for the parameters that underlie the model. The research suggests that the Burnham et al. study of conflict mortality in Iraq may represent a substantial overestimate of mortality. Indeed, the recently published Iraq Family Health Survey covered virtually the same time period as the Burnham et al. study, used census-based sampling techniques, and produced a central estimate for violent deaths that was one fourth of the Burnham et al. estimate. The authors provide a sensitivity analysis to help readers to tune their own judgements on the extent of this bias by varying the parameter values. Future progress on this subject would benefit from the release of high-resolution data by the authors of the Burnham et al. study.

Surviving the Storm: Behavioural Economics in the Conflict Environment

“…However, if one wants to obtain true behavioural insights from individuals in an extreme environment there is often no other choice but to go to the field. While the survey approach can be fruitful, one must be aware of the limitations of using such an approach in the estimation of casualties in conflict zones (Johnson et al 2008;Spagat 2012). 16 Analysis of one shot experiments or those that use cross sectional data do not suffer from participant attrition (drop outs), which is a major problem for research studies that run longitudinally and has the potential to create a significant bias in the results.…”

ETHICAL AND DATA‐INTEGRITY PROBLEMS IN THE SECONDLANCETSURVEY OF MORTALITY IN IRAQ

“…Burnham and Roberts have also issued a series of conflicting statements about their sampling procedures and have either destroyed or not collected evidence necessary to evaluate these procedures. Johnson et al (2008) suggests that sampling procedures described in L2 might have caused substantial upward bias in L2's estimate of the number of violent deaths. This idea is based on L2's published description of the final stages of its sampling methodology:…”

“…The published description goes on to explain that the field teams would then select a household on this residential cross street to a main street and then conduct interviews at 40 contiguous households. Johnson et al (2008) argues that residential cross streets to main streets would suffer from higher-than-average violence within the context of the Iraq War because:…”

“…Rather, I consider the responses of the L2 authors to the suggestion of possible sampling bias in L2. FIGURE 1 Figure 1 from Johnson et al (2008) illustrates the types of areas that will be missed by a methodology of conducting interviews at 40 contiguous households beginning at a household on a residential cross street to a main street. Scope is limited for reaching areas not actually on residential cross streets to main streets.…”

Modeling Human Conflict and Terrorism Across Geographic Scales

“…In addition to the traditional Blue (e.g., state military, terrorist group, or intelligence organization) and Red (e.g., insurgency or hacker group) actors, there is also a background civilian population which is labelled as Green, but which may not be passive in the struggle 11 Modeling Human Conflict and Terrorism Across Geographic Scales 213 may be cyber-defense, police, security forces, etc. Setting aside the issue of whether the data recorded has an observational bias or not due to the way it was recorded (e.g., main street bias [44]), there are many other potential complications facing a data-driven research program such as ours. These include, but are not limited to, the following: (1) Heterogeneity of the insurgent force strength (i.e., Red) which is depicted in Fig.…”