Equating With Miditests Using IRT

Abstract

The equating performance of two internal anchor test structures-miditests and minitests-is studied for four IRT equating methods using simulated data. Originally proposed by Sinharay and Holland, miditests are anchors that have the same mean difficulty as the overall test but less variance in item difficulties. Four popular IRT equating methods were tested, and both the means and SDs of the true ability of the group to be equated were varied. We evaluate equating accuracy marginally and conditional on true ability. Our results suggest miditests perform about as well as traditional minitests for most conditions. Findings are discussed in terms of comparability to the typical minitest design and the trade-off between accuracy and flexibility in test construction.

On the Choice of Anchor Tests in Equating

“…(, ) found minitests and semi‐miditests to lead to mostly similar equating (where both non‐IRT equating such as chain linear and equipercentile equating and IRT equating were considered) for data from the critical reading section of the SAT. Fitzpatrick and Skorupski () also found minitests and semi‐miditests, which were internal anchor tests, to lead to mostly similar transformation constants and similar equated ability estimates for simulated data. While most of this research focused on external anchor tests, Fitzpatrick and Skorupski () and Yi () considered internal anchor tests.…”

“…Fitzpatrick and Skorupski () also found minitests and semi‐miditests, which were internal anchor tests, to lead to mostly similar transformation constants and similar equated ability estimates for simulated data. While most of this research focused on external anchor tests, Fitzpatrick and Skorupski () and Yi () considered internal anchor tests. Together, the above‐mentioned studies demonstrated that equating using miditests and semi‐miditests is as satisfactory as or more satisfactory than equating using minitests for a variety of simulated and real data/tests and equating methods.…”

“…This note shows, on the basis of summary statistics reported for several operational tests of different types, that several conditions considered in the comparison of TLS16 are rarely seen in practice and hence their findings are inadequate to reverse the conclusions of Sinharay and Holland (, ) in the context of operational testing. So, the answer to the question asked above (“Should miditests continue to be considered in practice?”) seems to be “Yes.” Therefore, echoing the recommendations of Sinharay and Holland () and Fitzpatrick and Skorupski (), it is recommended that the choice of a minitest or a miditest or a semi‐miditest should be made depending on which of them leads to operational convenience because, the other factors being the same, the quality of equating would not depend on the choice of a minitest or a miditest or a semi‐miditest.…”

“…The above‐mentioned research implies that the requirement of an anchor test to have the same spread of item difficulty as the total tests may be too restrictive and need not be optimal, and that the primary motivator for using minitests or miditests or semi‐miditests should be the flexibility in test construction (Fitzpatrick & Skorupski, , p. 187). Especially, Sinharay and Holland () recommended enforcing a restriction on the spread of item difficulties of an anchor test only when it leads to operational convenience.…”

“…In contrast, Sinharay and Holland (), Liu et al. (, ), and Fitzpatrick and Skorupski () performed equating using simulated and real data including data from operational SAT, and those equating studies showed that miditests lead to equally good or slightly better equating than minitests in a wide range of conditions that occur in practice.…”