Was It Ethnic and Social-Class Bias or Statistical Artifact? Logical and Empirical Evidence against Freedle's Method for Reestimating SAT Scores

Wainer, Howard; Skorupski, William P.

doi:10.1080/09332480.2005.10722717

Cited by 11 publications

(13 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, in 2004 Dorans and Zeller wrote "there appears to be a DIF/difficulty relationship that merits some investigation" (p. 27). Furthermore, two subsequent papers have referred to Freedle's initial publication by describing and addressing exactly this statistical relationship: Wainer and Skorupski (2005) and Scherbaum and Goldstein (2008). We agree with them that it is an important and interesting scientific topic.…”

mentioning

confidence: 62%

Responding to Claims of Misrepresentation

Santelices

Wilson

2010

Harvard Educational Review

View full text Add to dashboard Cite

mentioning

confidence: 62%

Responding to Claims of Misrepresentation

Santelices

Wilson

2010

Harvard Educational Review

View full text Add to dashboard Cite

“…The study of differences in parameter b would allow us to explore Freedle’s phenomenon in the 3PL model and compare the results with those obtained from the Rasch model, which only allows for differences in parameter b . In addition, differences in parameter c aim to examine differences between the focal and the reference groups in the probability of low-ability students answering items correctly by chance, which is one of the hypotheses offered by researchers for the Freedle phenomenon (Wainer, 2009; Wainer & Skorupski, 2005). We did not investigate deeply differences in the parameter a as they were not readily interpretable and no accepted criteria to classify the size of the differences were found in the literature.…”

Section: Methodsmentioning

confidence: 99%

“…Previously, researchers have offered at least two types of accounts to explain this empirical finding: (a) the contention that the standardization approach does not sufficiently control for differences in the mean ability of the two compared distributions (Dorans, 2004; Schmitt & Bleistein, 1987), and (b) the guessing behavior observed in multiple-choice items (Kulick & Hu, 1989; Schmitt & Bleistein, 1987; Wainer, 2009; Wainer & Skorupski, 2005).…”

Section: Introductionmentioning

confidence: 99%

“…This statistical procedure was developed by Dorans and Kulick (1983). Freedle was criticized by Dorans and colleagues (Dorans, 2004; Dorans & Zeller, 2004a) and other researchers (Bridgeman & Burton, 2005; Wainer, 2009; Wainer & Skorupski, 2005), and his findings were attributed to the way Freedle applied the standardization approach, to the limitations of the standardization procedure, and to the role of guessing (among other issues). 3 When implementing the standardization approach appropriately, Dorans and Zeller (2004a) reported correlations somewhat smaller: a correlation of 0.27 between equated deltas and Mantel–Haenszel D-DIF and a correlation of 0.19 between equated deltas and the standardized formula score statistics for the African American/White comparison of Verbal items.…”

Section: Introductionmentioning

confidence: 99%

“…Wainer and colleagues (Wainer, 2009; Wainer & Skorupski, 2005) have also used guessing to explain the phenomenon described by Freedle. They claim that the two parts used in the standardization methodology (stratification on total SAT and drawing inferences from a division of items into two parts: easy and hard) are contradictory if you consider that students can answer a particular item correctly not only based on ability but also based on chance.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On the Relationship Between Differential Item Functioning and Item Difficulty

Santelices

Wilson

2011

Educational and Psychological Measurement

View full text Add to dashboard Cite

The relationship between differential item functioning (DIF) and item difficulty on the SAT is such that more difficult items tended to exhibit DIF in favor of the focal group (usually minority groups). These results were reported by Kulick and Hu, and Freedle and have been enthusiastically discussed by more recent literature. Examining the validity of the original reports of this systematic relationship is important so that we can move on to investigating more effectively its causes and the consequences associated to test score use. This article explores the hypothesis that the observed relationship between DIF and item difficulty observed in the SAT could be because of one of the following explanations: (a) the confounding of DIF and impact by the shortcomings of the standardization approach and/or (b) by random guessing. The relationship between DIF and item difficulty is examined using item response theory, which better controls for differences between impact and DIF than the standardization approach and also allows us to test the importance of guessing. The results obtained generally find evidence in support of the relationship between item difficulty and DIF suggesting that the phenomenon reported by earlier research is not a mere artifact of the statistical methodologies used to study DIF.

show abstract

College admission testing: Myths and realities in an age of admissions hype.

Camara

Correcting Fallacies About Educational and Psychological Testing.

View full text Add to dashboard Cite

and Sawtell (2003) for a review of high school grades and grade inflation. COLLEGE ADMISSION TESTING 149Copyright American Psychological Association. Not for further distribution.scores, two thirds of public colleges have reported that they are of considerable or moderate importance compared with just 57% of private colleges. In addition, a significantly lower percentage of New England colleges indicated admission tests were of considerable importance when compared with the rest of the nation (Hawkins & Clindinst, 2006). Zwick (2007) noted that colleges have consistently atttibuted considerable importance to grades and course rigor, but the percentage of institutions reporting considerable importance to admissions tests has increased since the mid-1990s.Admission tests are used for college entrance throughout many of the developed nations, and specialized instruments have been developed for admission to graduate schools as well as programs in medicine, law, dentistry, and business. However, it is the use of admission tests for entrance into undergraduate college that has received the overwhelming amount of criticism and media scrutiny since the 1970s.The ACT and SAT are taken by well over 2 million college-bound seniors who comprise about two thirds of all graduating high school students. The SAT as well as the Subject Tests, which are produced by the College Board and Educational Testing Service (ETS), and the ACT, developed by ACT, are among the most highly researched large-scale tests produced in the world. Of freshmen in 2003, 64.0% submitted SAT scores to 4-year colleges compared with 50.5% of students who submitted ACT scores. Hawkins and Clindinst (2006) stated that "students who enroll in private, 4-year colleges with high selectivity and low yield were the most likely to submit their SAT scores" (p. 40). The SAT, which was established several decades before the ACT, continues to be subjected to more scrutiny, probably because it is more popular among students and colleges on the East and West Coasts of the United States, where national media are concentrated, and because it is more often associated with students seeking admission to more selective colleges.In 2005, the SAT added the Writing Test to the Critical Reading and Mathematics tests and made other changes in the format of questions and content of the test to better reflect the instructional practices in schools and skills required for success in college (Milewski, Johnsen, Glazer, & Kubota, 2005). Similarly, the ACT, which includes subtests in reading, English, science, and math, also introduced an optional essay in 2005 and conducts curriculum surveys to ensure that the test remains aligned to content in high schools. The correlation between total scores for students who take both the ACT and SAT has been .89 to .92 in three separate concordance studies, which is similar to the magnitude of test-retest reliabilities reported for both tests (Schneider & Dorans, 1999). Both testing organizations have published substantial amounts of research on t...

show abstract

Was It Ethnic and Social-Class Bias or Statistical Artifact? Logical and Empirical Evidence against Freedle's Method for Reestimating SAT Scores

Cited by 11 publications

References 13 publications

Responding to Claims of Misrepresentation

Responding to Claims of Misrepresentation

On the Relationship Between Differential Item Functioning and Item Difficulty

College admission testing: Myths and realities in an age of admissions hype.

Contact Info

Product

Resources

About