Rasch analysis in physics education research: Why measurement matters

Abstract

This paper is part of the Focused Collection on Quantitative Methods in PER: A Critical Examination.] The Rasch model is a probabilistic model which describes the interaction of persons (test takers or survey respondents) with test or survey items and is governed by two parameters: item difficulty and person ability. Rasch measurement parallels physical measurement processes by constructing and using linear person and item measures that are independent of the particular characteristics of the sample and the test items along a unidimensional construct. The model's properties make it especially suitable for test construction and evaluation as well as the development and use of surveys. The evaluation of item fit with the model can pinpoint problematic items and flag idiosyncratic respondents. The possibility of determining sample-independent item difficulties makes it possible to use the Rasch model for linking tests and tracking students' progression. The use of the Rasch model in PER is continuously increasing. We provide an overview and examples of its use and benefits, and also outline common mistakes or misconceptions made by researchers when considering the use of the Rasch model. We focus in particular on the question of how Rasch modeling can improve some common practices in PER, such as test construction, test evaluation, and calculation of student gain on PER diagnostic instruments.

Rasch Analysis of Multitier Open-ended Light-Wave Instrument (MOLWI): Developing and Assessing Second-Years Sundanese-Scholars Alternative Conceptions

“…The Rasch analysis was originally made by Georg Rasch in 1960 to test construction in psychology with two types of parameters, a difficulty for each item and an ability for each person Masters, 1982;Rasch, 1960). Rasch analysis is a statistical method for describes the interaction of persons with test item that can be understood as a psychometric tool in social science and it has strong measurement properties (Planinic, Boone, Susac, & Ivanjek, 2019;Sumintono, 2018;Brandt, Moulton, & Duckor, 2015;Chan, Ismail, & Sumintono, 2014;Mui Lim, Rodger, & Brown, 2009;Joyce & Yates, 2007;Rasch, 1960) Furthermore, the various extensions of this relatively simple model have already been proposed since then and are regularly applied in assessments, such as in the field of health by Duncan, Bode, Lai, Perera, & Glycin (2003), in chemistry education by Herrmann-Abell, DeBoer, & Roseman (2009), in mathematics education by Long, Wendt, & Dunne (2011), and the emergence of Rasch measurement in language testing by McNamara & Knoch (2012). Rasch analysis is related to instrument analysis, thus it is appropriate to use in the field of education.…”

“…In the physics education context, Rasch analysis has ever been carried out for several studies including research by Kauertz & Fischer (2006), Planinic, Ivanjek, & Susac (2010), Oon & Subramaniam. (2013), Susac, Planinic, Klemencic, & Sipus (2018), and Planinic, Boone, Susac, & Ivanjek, (2019). While Rasch analysis that has been used to analysis of misconceptions in the physics learning includes Wind & Gale (2015) about, "Diagnostic Opportunities Using Rasch Measurement in the Context of a Misconceptions-Based Physical Science Assessment".…”

Criteria for collapsing rating scale responses: A case study of the CLASS

“…The instrument evaluation methods discussed in this paper are only small subset of the instrument evaluation methods developed by researchers in other disciplines (e.g., data science and psychometrics). We strongly recommend that instrument developers explore the space of classical test theory and item response theory to help improve assessment reliability and validity [30][31][32]. These tools can also test for potential biases in assessments across student demographics and courses contexts [33,34].…”

Editorial: Focused Collection: Quantitative Methods in PER: A Critical Examination

“…Other authors highlighted analytical approaches that push the boundaries of quantitative PER or that solve persistent and common issues in our analyses [15][16][17][18][19]. Finally, some authors showed that using multiple analytical approaches to address the same problem will strengthen the work we do [20][21][22].…”

“…instruments or survey items, as well as dispelled myths associated with this technique [21]. Battaglia et al used survey data to compare hierarchical and nonhierarchical cluster analysis and demonstrated the types of choices made with each method [22].…”