Understanding Protocol Performance: Impact of Criterion and Test Correlation

Abstract

The three criteria, loose, strict, and intermediate, have definite advantages and disadvantages over a large range of test correlations. Some of the advantages and disadvantages of the loose and strict criteria are impacted by test correlation. The advantages of the intermediate criteria are relatively independent of test correlation. When three or more tests are used in a protocol, consideration should be given to using an intermediate criterion, particularly if there is some test correlation. Greater test correlation diminishes the advantages of adding tests to a protocol, particularly with a loose or strict criterion. At higher test correlations, fewer tests in the protocol may be appropriate.

Understanding Protocol Performance: Impact of Test Performance

“…In general, the better the performance (higher d') of the individual tests, the better the performance of the protocol. This was evident in the first article (Turner, 2013) as when comparing the d' of the good-test (GT), fair-test (FT), and poor-test (PT) protocols. A more interesting question is the impact when individual tests vary in HT, FA, and d'.…”

“…his is the second of two articles concerning test protocols. It is recommended that the first article (Turner, 2013) be read prior to this article. These two articles examine the three factors that determine protocol performance: criterion, test correlation, and individual test performance.…”

“…The measure d' can be calculated from the HT and FA of an individual test or the HTp and FAp of a protocol. While d' has some limitations, as discussed in the first article (Turner, 2013), it is a convenient estimate of test and protocol performance.…”

“…The first article (Turner, 2013) examined the impact of criterion and test correlation on protocol performance. In that analysis, the individual tests were assumed to have the same performance (same HT, FA, and, thus, d').…”

“…The objective of this article is to examine the impact of test performance on protocol performance. It was evident in the first article (Turner, 2013) that the performance of the individual tests that make up a protocol has a significant impact on the performance of the protocol. In general, the better the performance (higher d') of the individual tests, the better the performance of the protocol.…”