Construct Maps as a Foundation for Standard Setting

Construct Maps in Standard Setting: A Response and Some Additional Thoughts

“…Other data added in subsequent rounds helps panelists further understand the meaning of possible cut-scores. This approach mirrors the exact steps outlined in the example in Wyse (2013). In my experience, the judgmental process using construct maps does not encounter several of the potential problems suggested in Skaggs (2013).…”

“…My hope is that after reading Wyse (2013), the commentaries, and this response, researchers and practitioners will see the potential benefits of using construct maps in various standard-setting applications and recognize the need for more research in these areas. I do hope, as Schulz (2013) put it, that my work "gets taken up and used by others in unimaginable ways" to improve standard setting.…”

“…I want to take a moment to thank Bunch (2013), Davis-Becker (2013), Kane and Tannenbaum (2013), Kingston, Tiemann, and Loughran (2013), McClarty (2013), Popham (2013), Schulz (2013), and Skaggs (2013), who wrote reflections on my article (Wyse, 2013) that appeared in a recent issue of Measurement: Interdisciplinary Research and Perspectives. The purpose of this article is to provide a response to some of the comments of those authors and to provide some additional thoughts on the use of construct maps in standard setting.…”

An Investigation of Undefined Cut Scores With the Hofstee Standard‐Setting Method

“…Green, Trimble, and Lewis () described a standard setting for a K‐12 test that used three standard‐setting methods. Likewise, Wyse () provided an example from a K‐12 program where contrasting groups and bookmark methods were used together. A particularly popular procedure is first using a test‐centered standard‐setting activity, conducting a compromise activity after that, and presenting the results of both to the advisory panel to make a recommendation.…”

“…An increasingly popular standard‐setting approach is to see if multiple methods converge on similar cut scores (see Davis‐Becker & Buckendahl, ; Wyse, ). A common use of multiple standard‐setting methods is using a test‐centered method, such as the bookmark (Lewis, Mitzel, & Green, ; Mitzel, Lewis, Patz, & Green, ) or Angoff () methods, or an examinee‐centered method, such as contrasting groups (Livingston & Zieky, ), along with a compromise method, such as the Hofstee () or Beuk () methods.…”