Assessing a Critical Aspect of Construct Continuity When Test Specifications Change or Test Forms Deviate from Specifications

Liu, Jinghua; Dorans, Neil J.

doi:10.1111/emip.12001

Cited by 13 publications

(21 citation statements)

References 21 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…He noted that his approach made sense when X and Y were measures of the same construct, but admitted that scaling test scores to grades or vice versa had issues. This brief report by Myers can be viewed as a remote harbinger of work on the population invariance of score linking functions done by Dorans and Holland (2000), Dorans (2004), Liu (2009), andDorans (2013).…”

Section: Fair Prediction Of a Criterionmentioning

confidence: 99%

“…Dorans (2004) used the population sensitivity of linking functions to assess score equity for two AP exams. Dorans, Liu, and Hammond (2008) SEA has also been used as a tool to evaluate score interchangeability when a test is revised (Liu & Dorans, 2013). Liu, Cahn, and Dorans (2006) and Liu and Walker (2007) used SEA tools to examine the invariance of linkages across the old and new versions of the SAT using data from a major field trail conducted in 2003.…”

Section: Fair Linking Of Test Scoresmentioning

confidence: 99%

“…By the 1980s, under the direction of its third president, Gregory Anrig, ETS established the industry standard for fairness assessment at the item level (Holland & Wainer, 1993;Zieky, 2011). This century, fairness analyses have begun to focus on relationships between tests that purport to measure the same thing in the same way across different subgroups (Dorans & Liu, 2009;Liu & Dorans, 2013).…”

mentioning

confidence: 99%

See 2 more Smart Citations

Ets Contributions to the Quantitative Assessment of Item, Test, and Score Fairness

Dorans

2013

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS R&D Scientific and Policy Contributions series, undergo a formal peerreview process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS R&D Scientific and Policy Contributions series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of ETS.The Daniel Eignor Editorship is named in honor of Dr. Daniel R. Eignor, who from 2001 until 2011 served the Research and Development division as Editor for the ETS Research Report series. The Eignor Editorship has been created to recognize the pivotal leadership role that Dr. Eignor played in the research publication process at ETS. i Abstract Quantitative fairness procedures have been developed and modified by ETS staff over the past several decades. ETS has been a leader in fairness assessment, and its efforts are reviewed in this report. The first section deals with differential prediction and differential validity procedures that examine whether test scores predict a criterion, such as performance in college, across different subgroups in a similar manner. The bulk of this report focuses on item level fairness, or differential item functioning, which is addressed in the various subsections of the second section. ETS Contributions to theIn the third section, I consider research pertaining to whether tests built to the same set of specifications produce scores that are related in the same way across different gender and ethnic groups. Limitations with the approaches reviewed here are discussed in the final section.Key words: fairness, differential prediction, differential item functioning, score equity assessment, ETS, quantitative methods ii ForewordSince its founding in 1947, ETS has conducted a significant and wide-ranging research program that has focused on, among other things, psychometric and statistical methodology; educational evaluation; performance assessment and scoring; large-scale assessment and evaluation; cognitive, developmental, personality, and social psychology; and education policy. This broadbased research program has helped build the science and practice of educational measurement, as well as inform policy debates.In 2010, we began to synthesize these scientific and policy contributions, with the intention to release a series of reports sequentially over the course of the next few years. These reports constitute the ETS R&D Scientific and Policy Contributions Se...

show abstract

Section: Fair Prediction Of a Criterionmentioning

confidence: 99%

Section: Fair Linking Of Test Scoresmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Ets Contributions to the Quantitative Assessment of Item, Test, and Score Fairness

Dorans

2013

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

show abstract

“…SEA has also been used as a tool to evaluate score interchangeability when a test is revised (Liu and Dorans 2013). Liu et al (2006) and Liu and Walker (2007) used SEA tools to examine the invariance of linkages across the old and new versions of the SAT using data from a major field trail conducted in 2003.…”

Section: Fair Linking Of Test Scoresmentioning

confidence: 99%

Contributions to the Quantitative Assessment of Item, Test, and Score Fairness

Dorans¹

2017

Methodology of Educational Measurement and Assessment

Self Cite

View full text Add to dashboard Cite

“…J. Liu and Dorans (2013) described how SEA can be used as a tool to assess a critical aspect of construct continuity, the equivalence of scores, whenever planned changes are introduced to testing programs. They also described how SEA can be used as a quality-control check to evaluate whether tests developed to a static set of specifications remain within acceptable tolerance levels with respect to equitability.…”

Section: Measures Of the Subpopulation Sensitivity Of Score Linking Fmentioning

confidence: 99%

Contributions to Score Linking Theory and Practice

Dorans

Puhan

2017

Methodology of Educational Measurement and Assessment

Self Cite

View full text Add to dashboard Cite

Assessing a Critical Aspect of Construct Continuity When Test Specifications Change or Test Forms Deviate from Specifications

Cited by 13 publications

References 21 publications

Ets Contributions to the Quantitative Assessment of Item, Test, and Score Fairness

Ets Contributions to the Quantitative Assessment of Item, Test, and Score Fairness

Contributions to the Quantitative Assessment of Item, Test, and Score Fairness

Contributions to Score Linking Theory and Practice

Contact Info

Product

Resources

About