Performance assessments have become popular in education and credentialing, and performance standards are common for interpreting and reporting scores. However, because of the unique characteristics of these assessments compared to multiple-choice tests (such as polytomous scoring), new and validstandard-setting methods are needed. Well-known standard-setting methods are no longer applicable. A number of promising methods for setting performance standards are described and their strengths and weaknesses are discussed. Suggestions for additional research are offered.

A study was conducted to evaluate four goodnessof-fit procedures using data simulation techniques. The procedures were evaluated using data generated according to three different item response theory models and a factor analytic model. Three different distributions of ability were used, as were three different sample sizes. It was concluded that the likelihood ratio chi-square procedure yielded the fewest erroneous rejections of the hypothesis of fit, whereas Bock's chisquare procedure yielded the fewest erroneous acceptances of fit. It was found that sample sizes somewhere between 500 and 1,000 were best. Shifts in the mean of the ability distribution were found to cause minor fluctuations, but they did not appear to be a major issue.

No abstract

No abstract

No abstract

This report summarizes the results from two studies. The first study assessed the comparability of scores derived from linear computer-based (CBT) and computer adaptive (CAT) versions of the three GRE General Test measures. The verbal and quantitative CATs were found to produce scores that were comparable to their CBT counterparts. However, the analytical CAT produced scores that were judged not to be comparable to the analytical CBT scores. As a result, a second study was performed to further examine the analytical measure to ascertain the extent of the lack of comparability and to obtain statistics that would permit adjustments to restore comparability.Results of the additional study of the analytical measure indicated that the differences in analytical CAT and CBT scores due to the testing paradigm were large enough to require an adjustment in scores. Therefore, in order to enhance the comparability of analytical CAT and CBT scores, the analytical CAT was equated to the analytical CBT. This equating provided new analytical CAT conversions that resulted in comparable analytical CAT and CBT scores.

No abstract

What factors influence judges when they set standards? How do judges, test questions, and the standard‐setting process interact? How can we improve intrajudge consistency?