Two-, three-, and four-interval forced-choice staircase procedures: Estimator bias and efficiency

Abstract

Threshold estimates for multiple-interval forced-choice staircase procedures were studied using computer simulations. A sigmoidal psychometric function shape governed the hypothetical subject's responses in the simulations. Parameters varied included the number of trials, the step size for stimulus level change, and decision rules that targeted 70.7% and 79.4% correct performance. Each threshold estimate was calculated by averaging the stimulus levels at which a reversal a stimulus level direction occurred. The results of the simulations suggest that, as the number of alternatives is increased from 2 to 4, the variability of repeated threshold estimates decreases or remains constant, and the accuracy of the estimator, in most cases, improves. A subset of the simulations was compared with data obtained in a detection-in-noise task. The behavioral data were consistent with the simulation results. Two major conclusions were reached. First, 3- and 4-interval forced-choice (IFC) procedures are more efficient than a 2IFC procedure with a decision rule that targets 70.7% correct performance even when the additional time required to complete 3- and 4IFC trials is considered. Second, the accuracy of 2IFC procedures can be improved by fitting the trial history of a staircase run using probit analysis.

Evaluation of maximum-likelihood estimators in nonintensive auditory psychophysics

“…The first four reversals were discarded, and threshold estimates were based on the average ofthe remaining reversals. Although we have used the average of reversals to obtain an estimate of threshold, other efficient rules of data summary could have also been used for this purpose (Schlauch & Rose, 1990;Watson & Fitzhugh, 1990). However, the present goal was to compare methods and rules that are commonly used in hearing research.…”

On the theoretical error bound for estimating psychometric functions

“…1 shows PFs corresponding to three different values of β (0.5, 1, 2) and a fixed value of α = 1. This form and similar forms of PFs have been used in previous studies (e.g., Buus et al, 1986;Dai, 1994Dai, , 1995Egan et al, 1969;Green & Swets, 1966;Leek et al, 1992;Schlauch & Rose, 1990).…”

“…Optimizing sampling strategy has been a major objective in previous efforts (e.g., Hall, 1968Hall, , 1981Hawley & Colburn, 1995;Jesteadt, 2005;King-Smith & Rose, 1997;Klein, 2001;Lam, Dubno, & Mills, 1999;Leek, 2001;Leek, Hanna, & Marshall, 1992;McKee, Klein, & Teller, 1985;Miller & Ulrich, 2001;O'Regan & Humbert, 1989;Schlauch & Rose, 1990;Taylor & Creelman, 1967;Watt & Andrews, 1981;Wichmann & Hill, 2001a, 2001b. The premise is that, given a fixed total number of trials for measuring a PF, of all the possible ways of selecting the signal levels and distributing the trials, there will be one implementation that will produce the most accurate estimate of the parameters, resulting in the least amount of error from the true values of the parameters.…”

Perceiving Patterns in Parallel Coordinates: Determining Thresholds for Identification of Relationships

“…Setting (P (x)) 2 = 0.5 gives P (x) = 0.707 which means that the two-up one-down staircase procedure will converge on that level of noise at which the participants can correctly identify the patterns with a certainty of 70.7 percent. This decision rule was applied in combination with the 3AFC response procedure, described above, which is recommended since forced-choice procedures with more than two alternatives provide more satisfactory measurements of psychometric performance [28]. For further details on this procedure, and the specific operational definitions described here and in the detailed methods section below, the interested reader is referred to, for example, Cornsweet [25], Leek [26] and Levitt [27].…”