Kernel Equating Using Propensity Scores for Nonequivalent Groups

Wallin, Gabriel; Wiberg, Marie

doi:10.3102/1076998619838226

Cited by 12 publications

(22 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the absence of anchor tests, then, collateral information has been used for test score linking either as surrogate anchors (Wallin & Wiberg, 2019) or to weight samples to approximate equivalent groups (Haberman, 2015; Livingston, 2014; Longford, 2015). These methods appear to work well when collateral information is not only abundant but also strongly statistically related to the test scores.…”

mentioning

confidence: 99%

Comparisons Among Approaches to Link Tests Using Random Samples Selected Under Suboptimal Conditions

Kim¹,

Walker²

2021

ETS Research Report Series

View full text Add to dashboard Cite

Equating the scores from different forms of a test requires collecting data that link the forms. Problems arise when the test forms to be linked are given to groups that are not equivalent and the forms share no common items by which to measure or adjust for this group nonequivalence. We compared three approaches to adjusting for group nonequivalence in a situation where not only is randomization questionable, but the number of common items is small. Group adjustment through either subgroup weighting, a weak anchor, or a mix of both was evaluated in terms of linking accuracy using a resampling approach. We used data from a single test form to create two research forms for which the equating relationship was known. The results showed that both subgroup weighting and weak anchor approaches produced nearly equivalent linking results when group equivalence was not met. Direct (random groups) linking methods produced the least accurate result due to nontrivial bias. Use of subgroup weighting and linking using the anchor test only marginally improved linking accuracy compared to using the weak anchor alone when the degree of group nonequivalence was small.

show abstract

mentioning

confidence: 99%

Comparisons Among Approaches to Link Tests Using Random Samples Selected Under Suboptimal Conditions

Kim¹,

Walker²

2021

ETS Research Report Series

View full text Add to dashboard Cite

show abstract

“…According to Wallin and Wiberg (2019), equating non-equivalent test groups requires adjusting for two sources of bias: differences in the difficulty of the forms and differences in the abilities of the test groups. A proper equating conversion should address both of these, but when the second is observed, some substitutes are required in place of ability.…”

Section: Data Set and The Test Equating Designmentioning

confidence: 99%

“…In the second step, the estimated score probabilities were generated by mapping the presmoothed score distributions into the score probability vectors for X and Y using a design function. This function, known as the design function, depends on the data collection design (see Wallin and Wiberg (2019) for the explicit expression of the design function for the NEC design).…”

Section: Estimation Of the Score Probabilitiesmentioning

confidence: 99%

Equality of admission tests using kernel equating under the non-equivalent groups with covariates design

Altıntaş

Wallin

2021

International Journal of Assessment Tools in Education

Self Cite

View full text Add to dashboard Cite

Educational assessment tests are designed to measure the same psychological constructs over extended periods. This feature is important considering that test results are often used for admittance to university programs. To ensure fair assessments, especially for those whose results weigh heavily in selection decisions, it is necessary to collect evidence demonstrating that the assessments are not biased and to confirm that the scores obtained from different test forms have statistical equality. Therefore, test equating has important functions as it prevents bias generated by differences in the difficulty levels of different test forms, allows the scores obtained from different test forms to be reported on the same scale, and ensures that the reported scores communicate the same meaning. In this study, these important functions were evaluated using real college admission test data from different test administrations. The kernel equating method under the non-equivalent groups with covariates design was applied to determine whether the scores that were obtained from different periods and measured the same psychological constructs were statistically equivalent. The non-equivalent groups with covariates design was specifically used because the test groups of the admission test are non-equivalent and there are no anchor items. Results from the analyses showed that the test forms had different score distributions and that the relationship was non-linear. Thus, the equating procedure was adjusted to eliminate these differences and thereby allowing the tests to be used interchangeably.

show abstract

“…Using operational data, Haberman concluded that equating with pseudoequivalent groups yielded results not identical with but similar to the operational results. Under the framework of kernel equating, Wallin & Wiberg (2019) examined the use of propensity scores for equating with nonequivalent groups when common items were unavailable and found the results satisfactory.…”

Section: No Samplementioning

confidence: 99%

Working with Atypical Samples

Cui¹

2020

Educational Measurement

View full text Add to dashboard Cite

Thanks to COVID‐19, schools were closed and tests were canceled. The result is that we may not see test‐taking data typically seen before. For some analyses, sample sizes may not meet the minimum requirement. For others, the sample of test‐takers may be different from previous years. In some situation, there may be no data at all. What do we do in these and other similar situations? Several ideas are presented in this article. Directions are suggested for not only dealing with challenges like this but also preparing for them.

show abstract

Kernel Equating Using Propensity Scores for Nonequivalent Groups

Cited by 12 publications

References 35 publications

Comparisons Among Approaches to Link Tests Using Random Samples Selected Under Suboptimal Conditions

Comparisons Among Approaches to Link Tests Using Random Samples Selected Under Suboptimal Conditions

Equality of admission tests using kernel equating under the non-equivalent groups with covariates design

Working with Atypical Samples

Contact Info

Product

Resources

About