Effects of Local Item Dependence on the Fit and Equating Performance of the Three-Parameter Logistic Model

Yen, Wendy M.

doi:10.1177/014662168400800201

Cited by 516 publications

(519 citation statements)

References 8 publications

Supporting

Mentioning

480

Contrasting

Unclassified

Order By: Relevance

“…Local independence of the DESIGMA items was tested using Yen's Q 3 statistic [47]. The latter tests the absence of meaningful pair-wise correlations of item residuals after controlling for person ability and item difficulty.…”

Section: Preliminary Checks and Data Preparationmentioning

confidence: 99%

Sometimes More Is Better, and Sometimes Less Is Better: Task Complexity Moderates the Response Time Accuracy Correlation

et al. 2016

View full text Add to dashboard Cite

This study addresses the relationship between item response time and item accuracy (i.e., the response time accuracy correlation, RTAC) in figural matrices tests. The dual processing account of response time effects predicts negative RTACs in tasks that allow for relatively automatic processing and positive RTACs in tasks that require controlled processing. Contrary to these predictions, several studies found negative RTACs for reasoning tests. Nevertheless, it was demonstrated that the RTAC is moderated by task complexity (i.e., the interaction between person ability and item difficulty) and that under conditions of high complexity (i.e., low ability and high difficulty) the RTAC was even slightly positive. The goal of this study was to demonstrate that with respect to task complexity the direction of the RTAC (positive vs. negative) can change substantially even within a single task paradigm (i.e., figural matrices). These predictions were tested using a figural matrices test that employs a constructed response format and has a broad range of item difficulties in a sample with a broad range of ability. Confirming predictions, strongly negative RTACs were observed when task complexity was low (i.e., fast responses tended to be correct). With increasing task complexity, the RTAC flipped to be strongly positive (i.e., slow responses tended to be correct). This flip occurred earlier for people with lower ability, and later for people with higher ability. Cognitive load of the items is suggested as an explanation for this phenomenon.

show abstract

Section: Preliminary Checks and Data Preparationmentioning

confidence: 99%

Sometimes More Is Better, and Sometimes Less Is Better: Task Complexity Moderates the Response Time Accuracy Correlation

et al. 2016

View full text Add to dashboard Cite

show abstract

“…Previous research has demonstrated the reasonableness of the three-parameter logistic model for the GRE verbal and quantitative measures (Kingston and Dorans, 1982a). Analysis of item-ability regressions (for an example of such an analysis see Kingston and Dorans, 1985) and a slightly modified Yen's Q 1 statistic (see Yen, 1981Yen, , 1984 indicated that the three-parameter logisfic model is probably reasonable for the current GRE General Test.…”

Section: Factors That May Have Affected These Resultsmentioning

confidence: 91%

Alternative Methods of Equating the Gre General Test

Kingston¹,

Holland²

1986

ETS Research Report Series

View full text Add to dashboard Cite

Financial support from the Graduate Record Examinations Board and Educational Testing Service is gratefully acknowledged. Our thanks to our many colleagues who made this research possible. The original plan for this research was designed by E. Elizabeth Stewart with the assistance of Madeline Wallmark and several other consultants. Many of the analyses were supervised or performed by Madeline Wallmark. Dorothy Thayer, and Craig Mills. We are especially grateful for the organizational and programming assistance of Louann Benton. We also thank Frederic Lord. Martha Stocking, and Marilyn Wingersky with whom we consulted many times and several colleagues who reviewed an earlier draft of this paper. We wish to thank especially E. Elizabeth Stewart for her insightful comments. Nonetheless, the opinions expressed herein are solely those of the authors. ABSTRACTThe original purpose of this study was to address the test-disclosure-related need to introduce more Graduate Record Examinations (GRE) General Test editions each year than formerly, in a context of stable, or possibly declining examinee volume. The legislative conditions that created this initial concern regarding test equating have abated. However, several of the test equating models considered in this research might provide other advantages to the GRE Program. These potential advantages are listed in the body of the report.Equating can be considered to consist of three parts: (1) a data collection design, (2) an operational definition of the equating transformation, and (3) the specific statistical estimation techniques used to obtain the equating transformation. Currently, the GRE General Test collects data using an equivalent groups design. Typically, a linear equating method is used, and the specific estimation technique is setting means and standard deviations equal.For this research, two other data collection designs Were studied: nonrandom group, external anchor test, and random group, preoperational section. Both item response theory (IRT) and linear equating definitions were used. IRT true score equating was based on item statistics for the three-parameter logistic model as estimated using LOGIST. Linear models included section pre-equating using the EM algorithm, Tucker's observed score model, and several true score models developed by Tucker and Levine. For each of the three GRE measures, verbal, quantitative and analytical, all equating methods were assessed for bias and root mean squared error by equating a test edition to itself through a chain with six equating links.Bias and root mean squared error were extremely large for equating the verbal and analytical measures using section pre-equating or IRT equating with data based on the random group preoperational section data collection design. For the quantitative measure, this data collection design produced a small amount of bias, but moderate amount of root mean squared error.Using the nonrandom group, external anchor test data collection design t quantitative equatings had moderate amounts of bo...

show abstract

“…Thus, the current study expands and improves the work of Thomas et al (2016) In the following sections, techniques that yield item properties that may be useful in detecting item compromise will be introduced and described including: Rasch model estimates, three-parameter logistic (3PL) IRT model estimates, item response times, Yen's (1984) Q3 for identifying local dependence of items, and linear logistic test model (LLTM) estimates. Then the methods for using these item properties and the suspected item categories to create functions that can separate compromised items from uncompromised items, which, in the current study, are Support Vector Machines (SVMs) will be discussed.…”

Section: List Of Figuresmentioning

confidence: 87%

“…Thomas et al (2016) suggest that examinees with pre-knowledge may have response patterns that are identifiable by their item response times and correctness of item responses. Examinees with pre-knowledge may respond more quickly to suspected compromised items than to other items, because they were previously exposed to these items, and thus may spend less time reading and responding to them (Marianti, Fox, Avetisyan, Veldkamp, & Tijmstra, 2014 Therefore, in the current study, Yen's Q3 statistic (Yen, 1984) for local dependence (LD) was calculated to use as an item feature that quantifies violations of this assumption.…”

Section: Item Response Timesmentioning

confidence: 99%

“…The item features calculated to classify suspected compromised and suspected uncompromised items into predicted compromised and predicted uncompromised items were: Rasch item difficulty, standard error, infit, and outfit, 3PL difficulty, discrimination, and lower limit, average item response times, the range of Yen's (1984) Q3 pairwise estimates for each item, and LLTM difficulty estimates. The relative importance of these item features in the SVMs was assessed.…”

Section: Item Features Summarymentioning

confidence: 99%

See 1 more Smart Citation

So Happy Together? Combining Rasch and Item Response Theory Model Estimates with Support Vector Machines to Detect Test Fraud

Thomas¹

View full text Add to dashboard Cite

Effects of Local Item Dependence on the Fit and Equating Performance of the Three-Parameter Logistic Model

Cited by 516 publications

References 8 publications

Sometimes More Is Better, and Sometimes Less Is Better: Task Complexity Moderates the Response Time Accuracy Correlation

Sometimes More Is Better, and Sometimes Less Is Better: Task Complexity Moderates the Response Time Accuracy Correlation

Alternative Methods of Equating the Gre General Test

So Happy Together? Combining Rasch and Item Response Theory Model Estimates with Support Vector Machines to Detect Test Fraud

Contact Info

Product

Resources

About