Item Response Theory-Based Evaluation of Psychometric Properties of the Safety Attitudes Questionnaire—Korean Version (SAQ-K)

Abstract

Years have passed since patient safety culture began to be measured. Several measurement instruments have been developed and used in the field, including the Safety Attitudes Questionnaire (SAQ). Validation is an essential step when administering these instruments to health care professionals. The problem is that confirmatory factory analysis (CFA) has mainly been used for the validation, despite the fact that CFA only works well with continuous responses. Yet SAQ and its variants, including the Korean version, use a 5-point Likert scale, which is by definition not a continuous, but a categorical measure. To resolve this conflict, we used item response theory (IRT) graded response model (GRM), an extension of CFA to categorical variables, to revalidate the SAQ-K and unravel the properties of each item and the SAQ-K instrument as a whole. We calculated difficulty and discriminating parameters for all items first. Then, based on the results, we estimated the expected score of each domain along the latent trait continuum, showing how the current SAQ behaves. We also used the itemlevel and domain-level information function, which can enlighten us on how to improve the precision of the current instrument. Most importantly, we obtained IRT-based empirical Bayes estimates of the latent trait level of each person and compared them with the traditional 0 to 100 SAQ arithmetic mean domain score scale. The correlation coefficients were very high, but when plotted we observed a considerable discrepancy between them. This finding led to the validity of the traditional scoring method in question. We expect the results and methodology of this study to bring about active discussions on the use of such safety culture instruments and help future studies on improving patient safety. Despite the successful application of such instruments in gauging patient safety culture in health care, we have to admit that those safety culture instruments, including SAQ-K, should be revalidated for several reasons. In most cases, the critical part of the survey questionnaire validation process has been conducted using confirmatory factor analysis (CFA) [19]. The problem here is that CFA can and should only be used for continuous responses, not for categorical responses such as a 5-point Likert scale, because CFA is basically a linear regression model. Therefore, it cannot appropriately deal with the errors in a measurement model for dichotomous or categorical responses-errors that can neither be normally distributed nor have constant variance. In addition, in the CFA paradigm, the probability of choosing an option for an item can go out of bounds [20]. Simply put, validating a Likert scale-based instrument with CFA is theoretically incorrect. Unfortunately, however, the original SAQ and all its variants, including SAQ-K, measure responses on a 5-point Likert scale (1= disagree strongly, 2= disagree slightly, 3= neutral, 4= agree slightly, 5= agree strongly) and then convert them to a scale from 0 to 100, with 25-point intervals. In all ho...

The Pure and the Overarching: An Application of Bifactor Model to Safety Attitudes Questionnaire

“…You do not have to understand the algorithm; just be aware that IRT scores follow a normal distribution with a mean of 0 and a standard deviation of 1 [15], which can be transformed to a 0 to 100 scale at any time. To learn more about IRT and its application to SAQ, please refer to our previous articles in this series [6,16].…”

A Novel Indicator for How Fairly Hospitals Treat Healthcare Professionals’ Safety Improvement Efforts: Differences in Hospital Responses to Medical Error Reports Depending the Type of Healthcare Professional Reporting the Event

“…Methodologies analyzing the survey responses have also evolved to get the most out of collected responses. For example, Jeong et al, in their series of articles, proposed several novel approaches to process and analyze the already collected safety culture questionnaire data and how to present them most effectively [12][13][14][15][16][17][18][19][20][21]. These efforts certainly helped us depict how various aspects of safety culture are intertwined in healthcare professionals' minds [22].…”

Untitled

“…Therefore, taking out less useful items could be one way to increase efficiency. Indeed, , using item response theory (IRT), proposed a method to reveal the amount of impact of each item on the safety culture estimatesin order to selectless influential items that can be removed from an instrument [3]. In this way, the survey can be administered to healthcare professionals with minimum burden.…”