Using biological data from field studies with multiple reference sites as a basis for environmental management: The risks for false positives and false negatives

An Adaptive Environmental Effects Monitoring Framework for Assessing the Influences of Liquid Effluents on Benthos, Water, and Sediments in Aquatic Receiving Environments

“…Further, given that EEM programs are designed to assess the effects of an intervention (i.e., effluent release) on an ecological response, the intervention will de facto cause change in the physical and chemical environment that will de facto result in change in biological responses no matter how trivial those changes are (Underwood 1994). The logic of EEM programs is, therefore, improved when programs are designed to test for changes of a prespecified magnitude, rather than testing for "any" change, including those that are trivially small (see also McBride et al 1993, Hanson 2011). The logic of EEM programs is, therefore, improved when programs are designed to test for changes of a prespecified magnitude, rather than testing for "any" change, including those that are trivially small (see also McBride et al 1993, Hanson 2011).…”

“…Baseline data for water, sediment, and biota collected during implementation of BA, CI, BACI, gradient, or RCA designs provide the data to derive monitoring (baseline) triggers. Data used to derive monitoring triggers based on the normal range typically originate from site-specific or regional reference areas lacking perturbation (Wissmar 1993;Roux et al 1999;Hanson 2011) or lacking influence from the intervention being assessed (i.e., effluent). The concept of the normal range has been adopted as a generic approach for deriving site-specific triggers for surveys of water quality (Chambers et al 2012), sediment quality (Thompson et al 2005;Kilgour et al 2017), and benthic macroinvertebrates Bailey et al 2004, Parsons et al 2010Environment Canada 2012).…”

“…Monitoring triggers are typically based on some measure of background variability or the normal range of variation of the monitoring data (Arciszewski et al 2017). The normal range can be estimated from a collection of random samples from a number of minimally influenced reference areas (Wissmar 1993;Roux et al 1999;Hanson 2011). Data used to derive monitoring triggers based on the normal range typically originate from site-specific or regional reference areas lacking perturbation (Wissmar 1993;Roux et al 1999;Hanson 2011) or lacking influence from the intervention being assessed (i.e., effluent).…”

“…For triggers based on reference-area data, special analytical approaches are required to demonstrate that the observed change was equal to or greater than the critical effect size (McBride et al 1993;Hanson 2011), potentially involving noncentral analysis of variance (Kilgour et al 1998(Kilgour et al , 2017Bowman and Somers 2006) or similar nonparametric procedures (Murphy 1948). In the Canadian EEM experience, the critical effect size for benthos is the estimated normal range for samples of reference-area data (i.e., x r ), where both the mean and SD are estimated with uncertainty.…”

“…As a result, the inability of an EEM program to detect a change is simply a problem of sample size (e.g., Osenberg et al 1994;Underwood 1994;Mapstone 1995) and is not related to whether or not a change occurred: a change did occur, regardless of how small and statistically undetectable the change was. The logic of EEM programs is, therefore, improved when programs are designed to test for changes of a prespecified magnitude, rather than testing for "any" change, including those that are trivially small (see also McBride et al 1993, Hanson 2011). Herein, we use the term "trigger" to refer to a value of a response variable that can be used to justify a continuation of the current monitoring tier or a change in monitoring tier (Arciszewski et al 2017).…”