AThurstonian Analysis of theTwo‐Out‐of‐Five Test

Abstract

The Two‐Out‐of‐Five test is a method of unspecified difference testing. Although its low guessing probability (1/10) gives promise that it might have high power, the theoretical underpinnings of the method have not yet been investigated. In this article, we offer the first such investigation, via Thurstonian analysis. This investigation reveals that the standard form of the Two‐Out‐of‐Five test is more statistically powerful than the Triangle test, but not as powerful as the Tetrad test. We then propose a new way of scoring Two‐Out‐of‐Five data that yields a test with higher power and lower sample size requirements than the Tetrad test, under the assumption that there is no additional noise from the evaluation of an additional stimulus. This last result is achieved without any experimental modification of the Two‐Out‐of‐Five protocol. Tables for estimating the Thurstonian measure of sensory effect size, δ, for calculating the error in such estimates, and for recommended sample sizes are given. Finally, caution is given against incorrect instructions in the Two‐Out‐of‐Five test – if respondents are asked simply to identify the two most similar samples, the resulting test has almost no power. Practical Applications This article shows that the standard form of the Two‐Out‐of‐Five test is more powerful than the Triangle test, but is not as powerful as the Tetrad test. The article then proposes a new method of scoring Two‐Out‐of‐Five data that requires smaller sample sizes than the Tetrad test, under the assumption that the evaluation of an additional stimulus does not lead to an increase in perceptual noise. When this alternate method of scoring the data is used, the test is called the Two‐Out‐of‐Five test with forgiveness. Tables for estimating δ, the Thurstonian measure of sensory difference, together with B values that allow practitioners to calculate the variance in their estimates are given for both forms of the Two‐Out‐of‐Five test. Tables of recommended sample sizes are also given in each case. Finally, it is noted that care must be taken to give correct instructions to respondents – it is possible to create a test with very low power if respondents are simply asked to identify the two most similar stimuli.

Estimation of Thurstonian Models for Various Forced‐Choice Sensory Discrimination Methods as a Form of the “M + N” Test

“…With the simulation‐derived psychometric functions, we can calculate a p c value for a d′ using the built‐in R/S‐Plus function “predict.” The p c = 0.211 for d′ = 1 for the unspecified two‐out‐of‐five test and p c = 0.633 for d′ = 1 for the 3‐AFC, while the corresponding p c = 0.209 in Ennis () and p c = 0.634 in ASTM () E2262‐03.…”

“…With the simulation‐derived psychometric function, we can obtain the d′ and the B values for any given p c by using the R/S‐Plus code “mndb.” For example, for p c = 0.634 for the 3‐AFC, d′ = 1.003 and B = 2.966, while d′ = 1 and B = 2.773 in ASTM () E2262‐03. For p c = 0.209 for the unspecified two‐out‐of‐five test, d′ = 1.024 and B = 4.051, while d′ = 1 and B = 3.938 in Ennis ().…”

“…The additional conditions (3) and (4) can avoid mistaken grouping. Ennis () discusses a special situation i.e., an unspecified two‐out‐of‐five test, where M ( m ) = 3, N ( n ) = 2, i.e., k = 1.…”

“…We can see that the fit is almost perfect. Figure shows the fit of the simulation‐derived psychometric function based on the smooth spline in this paper and based on the polynomial obtained in Ennis (). The results show high consistency.…”

“…(2013a,b) used a bootstrap method for the estimation of variances of the Thurstonian discriminal distance d′ and the area under an receiver operating characteristic (ROC) curve from signal detection theory. Ennis' () approach for the two‐out‐of‐five test used simulations to estimate the proportions of correct response ( P c ) for each of 500 equally spaced values of d′ , ranging from 0 to 3. He then used the estimated values to establish a polynomial function as an approximate psychometric function for the unspecified two‐out‐of‐five test.…”