Method of predicting reference condition biota affects the performance and interpretation of ecological indices

Abstract

1. The statistical rigour and interpretability of ecological assessments is strongly affected by how well we predict the biological assemblages expected to occur in the absence of human-caused stress, i.e. the reference condition. In this study, we examined how the specific method used to predict the reference condition affected the performance of two commonly used types of ecological index: RIVPACS-based O ⁄ E indices and multimetric indices (MMIs). 2. These two types of index have generally relied on different approaches to predicting the reference condition. For MMIs, some type of regionalisation is typically used to describe the range of metric values among reference sites and hence the expected range at assessed sites. For O ⁄ E indices, continuous modelling is used to predict how the biota varies among sites both among and within regions. Because the prediction method differs for these two types of index, it has been impossible to judge if differences in index performance (accuracy, precision, responsiveness and sensitivity) are caused by differences in the way reference condition biota are predicted or by differences in what the indices measure. 3. We used a common data set of 94 reference sites and 255 managed sites and the same potential set of predictor variables to compare the performance of five different MMIs and three O ⁄ E indices that were derived from different prediction methods: null models, multiple linear regression (MLR), classification and regression trees, Random Forests (RF) and linear discriminant functions models (LDM). We then calculated values of these indices for samples collected from the managed catchments as well as samples collected from 13 reference sites that were progressively altered in known ways by a simulation programme. 4. Both the type of predictor used and the type of index affected overall index performance. Modelled indices generally had the greatest sensitivity in assessing managed sites as biologically different from reference. Index sensitivity was determined by both an aspect of index precision (10th percentile of reference condition values) and responsiveness. The O ⁄ E indices showed the best scope of response to known biological alteration. All three O ⁄ E indices decreased linearly in response to simulated alteration in both overall assemblage structure (Bray-Curtis dissimilarity) and taxa loss. The MMIs declined linearly from low to intermediate levels of assemblage alteration but were less responsive between intermediate and high levels of biological alteration. 5. Insights gained from simulations can aid in testing assumptions regarding index response to stress and help ensure that we select indices that are ecologically interpretable and most useful to resource managers.

A new framework to evaluate ecosystem health: a case study in the Wei River basin, China

“…It was common that lower DO and higher TN and TP were measured in five reaches, which were located downstream of the main stem in the Wei River. From Fenggeling to confluence of the Wei River and Yellow River, big cities with large population density, such as Baoji, Xianyang, Xi'an, Weinan, were distributed from the upstream to the downstream, which caused great nutrient and organic pollution, such as industrial wastewater effluents and domestic disposal plant effluents, which brought much stressors to environment (Hawkins et al 2010). A similar result was found in the Taizi River (Wan et al 2012).…”

Accounting for regional variation in both natural environment and human disturbance to improve performance of multimetric indices of lotic benthic diatoms

“…In particular, the reduction in bias demonstrates the importance of accounting for natural variation in assessments Hawkins et al, 2010a;Schoolmaster et al, 2013). The bias that remained in EM-MMI1s and EM-MMI2s was due to not adjusting metrics if b 10% of the variation in original metrics was explained by site specific RF modeling.…”

“…We selected RF because it is a tree model based on non-parametric techniques that have been proven to perform better than other discrimination techniques such as linear discriminant analysis and generalized dissimilarity modeling (Cutler et al, 2007: Snelder et al, 2012Hawkins et al, 2010a). Ten naturally varying environmental factors were used as predictors, and the MC diatom typology group of each site was the response variable.…”

“…Performances of the five types of MMIs were compared with nine statistics grouped in five conceptual groups (Hawkins et al, 2010a;Stevenson et al, 2013;Vander Laan and Hawkins, 2014). (1) Precision was indicated by the coefficient of variance (CV) for MMIs among reference sites.…”

“…A multimetric index (MMI) is thereafter developed by using adjusted metrics that display significant differences between reference and highly disturbed sites. Site specific modeling has been increasingly used in lotic bioassessments and shown to increase performance of MMIs (Pont et al, 2006;Cao et al, 2007;Hawkins et al, 2010a).…”