The Usefulness of the Rasch Model for the Refinement of Likert Scale Questionnaires

Abstract

In this paper the use of the Rasch model is explored as a transparent, systematic and theoretically underpinned response to quality issues that are widely recognised as problematic in the refinement of Likert scale questionnaires. Key issues are the choice of length of scale, the pursuit of a favourable estimate of Cronbach's alpha at the possible expense of construct validity, and the fact that total raw scores arise from ordinal data but are used and interpreted as if measurement had occurred. We use a questionnaire under development for the measurement of perceptions of first-year chemistry students on demonstrator effectiveness to illustrate the process of Rasch analysis and instrument refinement. This process involves investigation of fit of the data to the model, possible violations of the assumption of local independence, and several aspects of item functioning. We identified disordered response categories as the probable reason for misfit in this data set and propose strategies for modification of items so that they can be retained rather than rejected.

Exploring the construct validity of the Patient Perception Measure – Osteopathy (PPM-O) using classical test theory and Rasch analysis

“…Fit of the individual items to the Rasch model is analysed to ascertain whether misfitting items are impacting upon the overall model fit. Poor individual item fit is indicated by a fit residual of ± 2.5 and/or a statistically significant chi-square probability [ 23 ]. In the case of the PPM-O it may be that the probability of a patient selecting a particular response on the Likert-type scale is not equal for all possible responses on the scale – this is referred to as a disordered threshold.…”

“…In the case of the PPM-O it may be that the probability of a patient selecting a particular response on the Likert-type scale is not equal for all possible responses on the scale – this is referred to as a disordered threshold. The threshold is the point at which there is a 50% chance of a person selecting response 1 or response 2 on a scale [ 21 , 23 ]. Where the threshold is disordered, respondents are selecting scale responses in a manner that is not consistent with the trait under investigation.…”

“…Misfit of individual persons to the Rasch model is indicated by a fit residual of ± 2.5 [ 23 ]. A person is said to misfit when their response to each of the item on a questionnaire, in this case on items on the PPM-O, does not follow the prediction of the Rasch model for how that person should have responded to the item.…”

“…The dimensionality of the measure is important because this demonstrates whether it is measuring a single underlying construct [ 26 ]. Local dependency is where the response to one item dictates the response to another item [ 23 , 26 ] and this can inflate the PSI [ 27 ]. Where local dependency is identified (the PSI decreases) one of the correlating items will need to be deleted.…”

Interactive Mobile Technologies on Civic Education Learning in Higher Education

“…The data obtained from the results of the response questionnaire of lecturers and students are then analyzed using quantitative data to test the practicality of the product developed. [47].…”