Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis

Abstract

Background: The System Usability Scale (SUS) is used to measure usability of internet-based Cognitive Behavioural Therapy (iCBT). However, whether the SUS is a valid instrument to measure usability in this context is unclear. The aim of this study is to assess the factor structure of the SUS, measuring usability of iCBT for depression in a sample of professionals. In addition, the psychometric properties (reliability, convergent validity) of the SUS were tested. Methods: A sample of 242 professionals using iCBT for depression from 6 European countries completed the SUS. Confirmatory Factor Analysis (CFA) was conducted to test whether a one-factor, two-factor, tone-model or bi-direct model would fit the data best. Reliability was assessed using complementary statistical indices (e.g. omega). To assess convergent validity, the SUS total score was correlated with an adapted Client Satisfaction Questionnaire (CSQ-3). Results: CFA supported the one-factor, two-factor and tone-model, but the bi-factor model fitted the data best (Comparative Fit Index = 0.992, Tucker Lewis Index = 0.985, Root Mean Square Error of Approximation = 0.055, Standardized Root Mean Square Residual = 0.042 (respectively χ 2 diff (9) = 69.82, p < 0.001; χ 2 diff (8) = 33.04, p < 0.001). Reliability of the SUS was good (ω = 0.91). The total SUS score correlated moderately with the CSQ-3 (CSQ1 r s = .49, p < 0.001; CSQ2 r s = .46, p < 0.001; CSQ3 r s = .38, p < 0.001), indicating convergent validity. Conclusions: Although the SUS seems to have a multidimensional structure, the best model showed that the total sumscore of the SUS appears to be a valid and interpretable measure to assess the usability of internet-based interventions when used by professionals in mental healthcare.

Usability evaluation of a virtual reality smartphone app for a living museum

“…Since then, several studies were conducted on whether it should be treated as unidimensional or bi-dimensional (i.e. [58][59][60]). Borsci et al [58] and Lewis and Sauro [59] proposed using a bi-dimensional analysis if the context of the study is suitable, and the participants are experienced users.…”

Organisational Implementation Climate in implementing internet-based Cognitive Behaviour Therapy for depression

“…Usability was measured with the System Usability Scale (SUS) using a 5-point Likert scale to rate 10 items [39,40]. It has good psychometric properties and is tested in numerous studies including samples of mental health professionals [39,41,42]. For calculating the SUS scale item's score contribution ranged from 0 to 4.…”

Development of Augmented Reality Application for Learning Computer Network Device

“…c) Data analysis using SUS Evaluation of AR application using the System Usability Scale: In this study, data analysis was performed using the SUS method. The System Usability Scale (SUS) method is a technique used to test the usability of an application [30]. SUS was developed as a "quick and dirty" usability measurement.…”