Goldilocks and the Raster Grid: Selecting Scale when Evaluating Conservation Programs

Abstract

Access to high quality spatial data raises fundamental questions about how to select the appropriate scale and unit of analysis. Studies that evaluate the impact of conservation programs have used multiple scales and areal units: from 5x5 km grids; to 30m pixels; to irregular units based on land uses or political boundaries. These choices affect the estimate of program impact. The bias associated with scale and unit selection is a part of a well-known dilemma called the modifiable areal unit problem (MAUP). We introduce this dilemma to the literature on impact evaluation and then explore the tradeoffs made when choosing different areal units. To illustrate the consequences of the MAUP, we begin by examining the effect of scale selection when evaluating a protected area in Mexico using real data. We then develop a Monte Carlo experiment that simulates a conservation intervention. We find that estimates of treatment effects and variable coefficients are only accurate under restrictive circumstances. Under more realistic conditions, we find biased estimates associated with scale choices that are both too large or too small relative to the data generating process or decision unit. In our context, the MAUP may reflect an errors in variables problem, where imprecise measures of the independent variables will bias the coefficient estimates toward zero. This problem may be pronounced at small scales of analysis. Aggregation may reduce this bias for continuous variables, but aggregation exacerbates bias when using a discrete measure of treatment. While we do not find a solution to these issues, even though treatment effects are generally underestimated. We conclude with suggestions on how researchers might navigate their choice of scale and aerial unit when evaluating conservation policies.

Mainstreaming Impact Evaluation in Nature Conservation

“…Selecting the appropriate scale can reduce the unobserved confounding factors, while using an inappropriate scale can exacerbate the effects of unobserved confounding factors. Where the appropriate scale is unknown, impact evaluations may use hierarchical models or replicate the analysis at multiple scales to evaluate how sensitive results are to the choice of scale (e.g., Avelino et al 2015;Börner et al 2015;Costedoat et al 2015).…”

Need for multiscale planning for conservation of urban bats

“…Yet, species perceive and respond to habitats at different scales (Levin 1992), and ecological studies in urban environments are often conducted at a single spatial scale (Pickett et al 2016). Conservation actions may be ill-informed and miss their mark if they do not account for species-specific scale dependencies (e.g., Graf et al 2005;Avelino et al 2016). Therefore, understanding the scale at which urban wildlife responds to the environment is critical for urban reconciliation.…”

Defining Landscapes and Scales to Model Landscape–Organism Interactions

“…Measurement error adds noise, uncertainty, can influence effect size [43], and might contribute to the MAUP [28]. With small sample sizes in particular, measurement error can increase the magnitude of the observed effect resulting in a significant relationship (type I error), whereas with larger samples the effect size would have been much smaller [43].…”

“…Often, researchers choose grain and extent subjectively, complicating comparisons between ecological studies [15]. The scale of spatial layers should match the data or processes of interest [58,59], and movement ecology can help us to identify those scales [28,60]. For example, researchers should not couple imprecise Argos satellite data with fine-scale habitat characteristics in a habitat-selection model [44]-similarly, the decision to use the PMM or GM should depend on the question at hand.…”

“…This can trigger the modifiable areal unit problem (MAUP), which remains an unresolved problem in landscape ecology. In short, MAUP occurs when the interpretation of results varies according to scale [28,29]. For example, a response variable can show a negative relationship with a predictor variable at a fine scale and a positive relationship at coarser scales [30].…”