Dangerous relationships: biases in freshwater bioassessment based on observed to expected ratios

Abstract

The ecological assessment of freshwaters is currently primarily based on biological communities and the reference condition approach (RCA). In the RCA, the communities in streams and lakes disturbed by humans are compared with communities in reference conditions with no or minimal anthropogenic influence. The currently favored rationale is using selected community metrics for which the expected values (E) for each site are typically estimated from environmental variables using a predictive model based on the reference data. The proportional differences between the observed values (O) and E are then derived, and the decision rules for status assessment are based on fixed (typically 10th or 25th) percentiles of the O/E ratios among reference sites. Based on mathematical formulations, illustrations by simulated data and real case studies representing such an assessment approach, we demonstrate that the use of a common quantile of O/E ratios will, under certain conditions, cause severe bias in decision making even if the predictive model would be unbiased. This is because the variance of O/E under these conditions, which seem to be quite common among the published applications, varies systematically with E. We propose a correction method for the bias and compare the novel approach to the conventional one in our case studies, with data from both reference and impacted sites. The results highlight a conceptual issue of employing ratios in the status assessment. In some cases using the absolute deviations instead provides a simple solution for the bias identified and might also be more ecologically relevant and defensible.

What’s wrong with the Australian River Assessment System (AUSRIVAS)?

“…Finally, because the value of the AUSRIVAS O/E index depends on the number of expected taxa that are recorded in a sample, the index value is highly sensitive to the chance detection or non-detection of individual taxa that are present at an assessment site (Smith et al 1999). This issue is particularly acute for naturally harsh environments with low taxon richness, such as dryland or nutrient-deficient streams or the profundal zone of lakes (Chessman et al 2006;Halse et al 2007;Jyväsjärvi et al 2011), because the intrinsic variability of O/E is higher when the number of expected taxa is low (Hämäläinen et al 2018).…”

Phylogenetic diversity efficiently and accurately prioritizes conservation of aquatic macroinvertebrate communities

“…They are less sensitive to degradation in heterogeneous environments (Hargett et al 2007), and uncertainty associated with expectations of potential invertebrates present is often not reflected in the final O/E metric (but see de Zwart et al 2006). Further, standards for what constitutes a degraded O/E are often subjective, and statistical deviation from this baseline can be difficult to determine (Hämäläinen et al 2018). The problem is even more acute given variation (and uncertainty; see Introduction) in expectations of site richness: Sites may have the same O/E value even when they have drastically different values of E. For instance, a site with an of and another with an of represent the same O/E value of 0.8, but it is unclear whether this value conveys the same information about species losses for both sites.…”

Phylogenetic diversity efficiently and accurately prioritizes the conservation of aquatic macroinvertebrate communities

“…They are less sensitive to degradation in heterogeneous environments (Hargett, ZumBerge, Hawkins, & Olson, 2007), and uncertainty associated with expectations is often not reflected in the final O/E metric (but see de Zwart, Dyer, Posthuma, & Hawkins, 2006). Further, standards for what constitutes a degraded O/E are often subjective, and statistical deviation from this baseline can be difficult to determine (Hamäläinen, Aroviita, Jyväsjärvi, & Kärkkäinen, 2018). The problem is even more acute given variation (and uncertainty; see above) in expectations of site richness: sites may have the same O/E value even when they have drastically different values of E. For instance, a site with an of and another with an of represent the same O/E value of 0.8, but it is unclear whether this value conveys the same information for both sites.…”