An approach for determining bioassessment performance and comparability

Abstract

Many organizations in the USA collect aquatic bioassessment data using different sampling and analysis methods, most of which have unknown performance in terms of data quality produced. Thus, the comparability of bioassessments produced by different organizations is often unknown, ultimately affecting our ability to make comprehensive assessments on large spatial scales. We evaluated a pilot approach for determining bioassessment performance using macroinvertebrate data obtained from several states in the Southeastern USA. Performance measures evaluated included precision, sensitivity, and responsiveness to a human disturbance gradient, defined in terms of a land disturbance index value for each site, combined with a value for specific conductance, and instream habitat quality. A key finding of this study is the need to harmonize ecoregional reference conditions among states so as to yield more comparable and consistent bioassessment results. Our approach was also capable of identifying potential areas for refinement such as reevaluation of less precise, sensitive, or responsive metrics that may result in suboptimal index performance. Higher performing bioassessments can yield information beyond "impaired" versus "unimpaired" condition. Acknowledging the limitations of this pilot study, we would recommend that performance evaluations use at least 50 sites, 10 of which are ecoregional reference sites. Efforts should be made to obtain data from the entire human disturbance gradient in an ecoregion to improve statistical confidence in performance measures. Having too few sites will result in an under-representation of certain parts of the disturbance gradient (e.g., too few poor quality sites), which may bias sensitivity and responsiveness estimates.

Development of an Index of Biotic Integrity for the Sand Hills Ecoregion of the Southeastern United States

“…The SHIBI CVs of 12.5% (project data set) and 11.9% (independent data set 1) were comparable to the 7% and 12%, respectively, reported for fish and macroinvertebrate-based multimetric indices for the South Carolina Coastal Plain (Paller 2001) and the 12-13% reported for the macroinvertebrate-based Florida stream condition index (Gerritsen et al 2000). Although we could find no comparable data for fish, the responsiveness (R 2 ) of 0.28 exhibited by the SHIBI was congruent with the finding by Diamond et al (2012) that responsiveness was under 0.30 for most BMI-based indices developed for the southeastern United States. Statistical power analysis showed that only moderate to small numbers of samples were needed to detect comparatively small differences (i.e., 3 or 4 index points) in SHIBI scores.…”

“…The performance of bioassessment indices is affected by their variability, sensitivity, responsiveness, accuracy, and statistical power (Diamond et al 1996(Diamond et al , 2012, as explained below.…”

“…Responsiveness.-Responsiveness, the degree to which an IBI changes monotonically with disturbance (Diamond et al 2012), was assessed by calculating the coefficient of determination (R 2 ) between the IBI scores and the instream habitat quality scores for the project data set. Responsiveness was further evaluated by assessing the extent to which the IBI scores differed between disturbed sites and reference sites.…”

“…The PCA gradient that was used to rank the sites provided a comprehensive, integrative, and relatively objective approach for characterizing disturbance, but it may not have included all salient factors or it may have weighted some factors suboptimally. Diamond et al (2012) indicated that bioassessment performance is primarily dependent on the error associated with the protocol and on the error caused by misclassifying sites on the human disturbance gradient.…”

Biological assessment of streams and rivers in U.S. - design, methods, and analysis

“…The selection of metrics is a well-documented activity (e.g., Barbour et al 1999, Whittier et al 2007). Metrics should be tested for performance across the full human disturbance gradient in a region, in order to improve statistical confidence, and to reduce bias (Diamond et al 2012). A multivariate approach has also been used to analyze bioassessment data.…”