REVISING SAT^®‐VERBAL ITEMS TO ELIMINATE DIFFERENTIAL ITEM FUNCTIONING

Abstract

Based on initial SAT‐Verbal pretest data and/or hypotheses advanced in the research literature, the authors selected 7 sentence completions and 16 analogies with extreme levels of differential item functioning (DIF) and then systematically revised and readministered the items in an attempt to reduce or eliminate DIF. Several diverse conclusions can be drawn from the data. First, because of the apparent success in reducing extreme levels of DIF in SAT‐Verbal items, the authors recommend that such efforts be continued. Second, the particular terminology used in stems and keys (rather than the underlying reasoning skill being measured) seems to be a recurring source of DIF in SAT‐Verbal items. Third, larger sample sizes, particularly for minority focal groups, would help to stabilize the DIF categories used by Educational Testing Service (ETS) test developers. Fourth, because the ETS delta metric is unbounded at the extremes, the use of both the Standardization (p‐metric) and Mantel‐Haenszel (delta‐metric) methodologies is recommended for classifying the level of DIF for very easy and very difficult items. Finally, the paper concludes with a suggestion for further research concerning the possible relationship between DIF and predictive validity.

Causes of gender DIF on an EFL language test: A multiple-data analysis over nine years

“…This finding is consistent with those of Lawrence and colleagues (Lawrence & Curley, 1989;Lawrence et al, 1988) but inconsistent with that of previous studies reporting a close relationship between gender DIF and item content (e.g. Carlton & Harris, 1992;Curley & Schmitt, 1993;Lawrence & Curley, 1989;. This inconsistency is therefore worthy of further attention in future research on gender DIF.…”

Who's on First? Gender Differences in Performance on the SAT® Test on Critical Reading Items With Sports and Science Content

“…A word or phrase that might come across as confusing or specialized in a 15-to 30-word SNCP is made less so when used in the context of a full reading passage-and items associated with the longest reading passages exhibit the smallest differences in performance among matched groups of males and females, regardless of perceived specialization of topic. This finding is in keeping with prior research on gender differences in performance, much of which was conducted on antonym and analogy item types, which used even fewer words and provided even less context than SNCPs (see Carlton & Harris, 1992;Curley & Schmitt, 1993;Schmitt & Bleistein, 1987).…”

Examining the Relationship of Content to Gender‐based Performance Differences in Advanced Placement Exams

“…For both sentence completion and reading comprehension items, Carlton and Harris report that females performed relatively better than males when people were referred to in the items as compared to when there was no reference to people. Curley and Schmitt (1993) reported that "science, industrial arts, and military terminology, as well as contexts portraying aggression or conflict may negatively affect the performance of females" (p. 2). O'Neill and McPeek (1993) largely concurred: They suggested that on sentence completion and analogy items related to science and practical affairs, males performed better than females, whereas on aesthetics/philosophy and human relationships, females performed better than males.…”