What is the best index of detectability?

Simpson, Adrian; Fitter, M. J.

doi:10.1037/h0035203

Cited by 191 publications

(135 citation statements)

References 10 publications

(9 reference statements)

Supporting

Mentioning

129

Contrasting

Order By: Relevance

“…Because the vertical distance to the chance line varies along the ROC curve, a decision must be made about the point at which "sensitivity" is to be defined or, equivalently, how the two standard deviations are to be combined. We follow Donaldson (1993) in adopting d a (Simpson & Fitter, 1973), which measures the mean difference in units of the root mean square of the two standard deviations. Pollack and Norman (1964) as the average of the maximum and minimum areas of ROCs containing the point (F, H ).…”

Section: Calculational Methodsmentioning

confidence: 99%

Measures of sensitivity based on a single hit rate and false alarm rate: The accuracy, precision, and robustness of′,A z, andA’

Verde

Macmillan

Rotello

2006

Perception & Psychophysics

115

View full text Add to dashboard Cite

Section: Calculational Methodsmentioning

confidence: 99%

Measures of sensitivity based on a single hit rate and false alarm rate: The accuracy, precision, and robustness of′,A z, andA’

Verde

Macmillan

Rotello

2006

Perception & Psychophysics

115

View full text Add to dashboard Cite

“…The sensitivity parameter P(A), that is, the area under the receiver-operating characteristic (ROC) curve, was the principal performance measure (since it has the advantage of being relatively independent of the shapes and variances of the underlying distributions of noise and signal plus noise; see Green & Swets, 1966;Simpson & Fitter, 1973). In both stages of analysis, computerdetermined estimates were obtained for the P(A) parameter and its variance andfor thelikelihood ratio beta at theyes-no cutoff point.…”

Section: Methodsmentioning

confidence: 99%

Sensitivity and criterion effects in the spatial cuing of visual attention

Müller

Findlay

1987

Perception & Psychophysics

257

179

View full text Add to dashboard Cite

This study is concerned with three questions about the role of attention in peripheral detection: (1) Is the increased detection rate for spatial locations with high probabilities of target occurrence assigned to them due to sensitivity or criterion effects? (2) Does the effect of spatial cuing (probabilistic priming) require different explanations for letter detection and detection of luminance increments (Shaw, 1984)? (3) Can attention be shared between two separate locations cued to be most likely (Posner, Snyder, & Davidson, 1980)? These questions were investigated in two experiments, both using a signal detection plus localization task (rating method). In Experiment 1 (symbol detection), single or double cues indicating one or two most likely locations (three or two least likely locations) were presented. Introducing the second cued location resulted in a marked sensitivity gain for this position, relative to uncued locations in the single-cue condition. Decision criteria were more liberal for cued and more conservative for uncued locations. In Experiment 2, a luminance increment (single target probe) and two symbol detection (target plus distractors) tasks were compared. For symbol detection, there was a marked priming effect; but for luminance detection, cued locations showed no advantage in sensitivity. However, all tasks showed differential criterion setting for cued and for uncued locations. These results suggest that letter detection is capacity limited, whereas luminance increment detection is not, and furthermore, that decision criteria are largely preset according to a priori target probabilities assigned to particular locations.How does the focusing and dividing of attention influence perception? Recent research on this question has been concerned with two effects: that of "display N," that is, the number of nontargets in the display, and that of "probabilistic priming," that is, the relative frequencies of target occurrence assignedto individuallocations. Experimentswith accuracy as the dependent measure and with high error rates have established that an increase in the number of nontargets, withoutan increase in the number of targets, reduces target detectability (e.g., Estes & Taylor, 1964), and that variations in the probabilitieswith whichthe target occurs at particularlocationsenhancetarget detectability for the more likely positions relative to the less likely locations (e.g., M. L. Shaw & P. Shaw, 1977).Equivalenteffects havebeen demonstratedin reaction time (RT) For the explanation of these effects, it seems useful to assume that there are at least two functional stages between stimulus and response: In the first, "coding," stage, which may consist of a number of substages, each stimulus is converted into an internal representation. In the second, "decision," stage, the internal representation is used to determine a response, for example, a "target present-absent" decision. According to signal detection theory (SDT; Green & Swets, 1966), the internal stimulus representation is characte...

show abstract

“…Equivalent expressions to ܸ have proposed in the ROC literature (e.g., Simpson and Fitter, 1973), where it is commonly known as ݀ . However, in the context of medical tests, AUC-type measures are most commonly used (Pepe, 2003).…”

Section: Estimating Achievement Gapsmentioning

confidence: 99%

Estimating Achievement Gaps From Test Scores Reported in Ordinal “Proficiency” Categories

Ho¹,

Reardon

2012

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

Test scores are commonly reported in a small number of ordered categories. Examples of such reporting include state accountability testing, Advanced Placement tests, and English proficiency tests. This paper introduces and evaluates methods for estimating achievement gaps on a familiar standard-deviation-unit metric using data from these ordered categories alone. These methods hold two practical advantages over alternative achievement gap metrics. First, they require only categorical proficiency data, which are often available where means and standard deviations are not. Second, they result in gap estimates that are invariant to score scale transformations, providing a stronger basis for achievement gap comparisons over time and across jurisdictions. We find three candidate estimation methods that recover full-distribution gap estimates well when only censored data are available. Researchers selecting an achievement gap metric face three issues. First, average-based gaps-effect sizes or simple differences in averages-are variable under plausible transformations of the test score scale (Ho, 2007; Reardon, 2008a;Seltzer, Frank, & Bryk, 1994;Spencer, 1983). Second, gaps based on percentages above a cut score, such as differences in "proficiency" or passing rates, vary substantially under alternative cut scores (Ho, 2008;Holland, 2002). Third, researchers often face a practical challenge: Although they may wish to use an average-based gap metric, the necessary data may be unavailable.This last situation has become common even as the reporting requirements of the No Child Left Behind Act (NCLB) have led to large amounts of easily accessible test score data.The emphasis of NCLB on measuring proficiency rates over average achievement has led states and districts to report "censored data": test score results in terms of categorical achievement levels, typically given labels like "below basic," "basic," "proficient," and "advanced. Traditional Achievement Gap Measures and Their ShortcomingsA test score gap is a statistic describing the difference between two distributions.Typically, the target of inference is the difference between central tendencies. Three "traditional" gap metrics dominate this practice of gap reporting. The first is the test score scale,where gaps are most often expressed as a difference in group averages. For a student test score, ܺ, a typically higher scoring reference group, ܽ, and a typically lower scoring focal group, ܾ, the difference in averages, ݀ ௩ , follows:The second traditional metric expresses the gap in terms of standard deviation units. This metric allows for standardized interpretations when the test score scale is unfamiliar and affords aggregation and comparison across tests with differing score scales (Hedges & Olkin, 1985).Sometimes described as Cohen's ݀, this effect size expresses ݀ ௩ in terms of a quadratic average of both groups' standard deviations, ‫ݏ‬ and ‫ݏ‬ . Although a weighted average of variances or a single standard deviation could also be used in the denomina...

show abstract

What is the best index of detectability?

Cited by 191 publications

References 10 publications

Measures of sensitivity based on a single hit rate and false alarm rate: The accuracy, precision, and robustness of′,A z, andA’

Measures of sensitivity based on a single hit rate and false alarm rate: The accuracy, precision, and robustness of′,A z, andA’

Sensitivity and criterion effects in the spatial cuing of visual attention

Estimating Achievement Gaps From Test Scores Reported in Ordinal “Proficiency” Categories

Contact Info

Product

Resources

About