Test of significant toxicity: A statistical application for assessing whether an effluent or site water is truly toxic

Abstract

The U.S. Environmental Protection Agency (U.S. EPA) and state agencies implement the Clean Water Act, in part, by evaluating the toxicity of effluent and surface water samples. A common goal for both regulatory authorities and permittees is confidence in an individual test result (e.g., no-observed-effect concentration [NOEC], pass/fail, 25% effective concentration [EC25]), which is used to make regulatory decisions, such as reasonable potential determinations, permit compliance, and watershed assessments. This paper discusses an additional statistical approach (test of significant toxicity [TST]), based on bioequivalence hypothesis testing, or, more appropriately, test of noninferiority, which examines whether there is a nontoxic effect at a single concentration of concern compared with a control. Unlike the traditional hypothesis testing approach in whole effluent toxicity (WET) testing, TST is designed to incorporate explicitly both α and β error rates at levels of toxicity that are unacceptable and acceptable, given routine laboratory test performance for a given test method. Regulatory management decisions are used to identify unacceptable toxicity levels for acute and chronic tests, and the null hypothesis is constructed such that test power is associated with the ability to declare correctly a truly nontoxic sample as acceptable. This approach provides a positive incentive to generate high-quality WET data to make informed decisions regarding regulatory decisions. This paper illustrates how α and β error rates were established for specific test method designs and tests the TST approach using both simulation analyses and actual WET data. In general, those WET test endpoints having higher routine (e.g., 50th percentile) within-test control variation, on average, have higher method-specific α values (type I error rate), to maintain a desired type II error rate. This paper delineates the technical underpinnings of this approach and demonstrates the benefits to both regulatory authorities and permitted entities.

Evaluation of whole effluent toxicity data characteristics and use of Welch's T‐test in the test of significant toxicity analysis

“…Thus, a slight increase in the type I error rate using Welch's t test and nonnormal, heterogeneous data would have little effect on the type II error rate at a 10% or less mean effect. As noted in previous U.S. EPA guidance [1] and other publications [4,5] regarding the TST approach, the rate at which a sample is declared toxic will increase as the mean effect increases above 10%. Thus, the type II error rate has been controlled in the TST approach, as discussed in previous publications [1,4].…”

“…As noted in previous U.S. EPA guidance [1] and other publications [4,5] regarding the TST approach, the rate at which a sample is declared toxic will increase as the mean effect increases above 10%. Thus, the type II error rate has been controlled in the TST approach, as discussed in previous publications [1,4]. Therefore, so long as the type I error rate remains stable with different types of data distributions or unequal variances, as demonstrated in the present study, very little change in the type II rate with percent mean effect is expected.…”

“…The higher a levels include WET test methods that have smaller sample sizes such as the fathead minnow acute test. The slight overestimation of the nominal type I error rate that can occur using Welch's t test when WET test data are strongly nonnormally distributed is insignificant given the higher nominal a levels established for these methods [1,4]. For the West Coast WET test methods that have a levels set at 0.05, effect size examined in those test methods is large and, in many cases, data are normally distributed even without data transformation (e.g., giant kelp germination and germ-tube length endpoints, Table 2).…”

“…The test of significant toxicity (TST) was recently proposed as an alternative statistical approach for analyzing WET data. The TST approach is a statistical method that uses hypothesis-testing techniques based on previous U.S. Environmental Protection Agency (U.S. EPA) guidance [1] as well as work by many researchers [2][3][4][5]. The TST examines whether the results of two treatments differ by an a priori prescribed amount rather than whether they are the same, as in traditional hypothesis testing [1,4].…”

“…The TST approach is a statistical method that uses hypothesis-testing techniques based on previous U.S. Environmental Protection Agency (U.S. EPA) guidance [1] as well as work by many researchers [2][3][4][5]. The TST examines whether the results of two treatments differ by an a priori prescribed amount rather than whether they are the same, as in traditional hypothesis testing [1,4]. The TST approach for WET testing uses the null hypothesis: m T b * m C. This null hypothesis includes a specific value for the ratio m T /m C , designated b (where b is a constant, 0.0 < b < 1.0), to delineate unacceptable and acceptable levels of toxicity.…”