Validation of the Raw National Aeronautics and Space Administration Task Load Index (NASA-TLX) Questionnaire to Assess Perceived Workload in Patient Monitoring Tasks: Pooled Analysis Study Using Mixed Models

Abstract

Background Patient monitoring is indispensable in any operating room to follow the patient’s current health state based on measured physiological parameters. Reducing workload helps to free cognitive resources and thus influences human performance, which ultimately improves the quality of care. Among the many methods available to assess perceived workload, the National Aeronautics and Space Administration Task Load Index (NASA-TLX) provides the most widely accepted tool. However, only few studies have investigated the validity of the NASA-TLX in the health care sector. Objective This study aimed to validate a modified version of the raw NASA-TLX in patient monitoring tasks by investigating its correspondence with expected lower and higher workload situations and its robustness against nonworkload-related covariates. This defines criterion validity. Methods In this pooled analysis, we evaluated raw NASA-TLX scores collected after performing patient monitoring tasks in four different investigator-initiated, computer-based, prospective, multicenter studies. All of them were conducted in three hospitals with a high standard of care in central Europe. In these already published studies, we compared conventional patient monitoring with two newly developed situation awareness–oriented monitoring technologies called Visual Patient and Visual Clot. The participants were resident and staff anesthesia and intensive care physicians, and nurse anesthetists with completed specialization qualification. We analyzed the raw NASA-TLX scores by fitting mixed linear regression models and univariate models with different covariates. Results We assessed a total of 1160 raw NASA-TLX questionnaires after performing specific patient monitoring tasks. Good test performance and higher self-rated diagnostic confidence correlated significantly with lower raw NASA-TLX scores and the subscores (all P<.001). Staff physicians rated significantly lower workload scores than residents (P=.001), whereas nurse anesthetists did not show any difference in the same comparison (P=.83). Standardized distraction resulted in higher rated total raw NASA-TLX scores (P<.001) and subscores. There was no gender difference regarding perceived workload (P=.26). The new visualization technologies Visual Patient and Visual Clot resulted in significantly lower total raw NASA-TLX scores and all subscores, including high self-rated performance, when compared with conventional monitoring (all P<.001). Conclusions This study validated a modified raw NASA-TLX questionnaire for patient monitoring tasks. The scores obtained correctly represented the assumed influences of the examined covariates on the perceived workload. We reported high criterion validity. The NASA-TLX questionnaire appears to be a reliable tool for measuring subjective workload. Further research should focus on its applicability in a clinical setting.

Feasibility of Cognitive-Motor Exergames in Geriatric Inpatient Rehabilitation: A Pilot Randomized Controlled Study

“…For evaluation, the raw NASA-TLX was used which is an average workload score between 1 and 100 calculated by multiplying each rating by 5. Additionally, an overall workload score was calculated averaging the ratings on the six subscales ( Said et al, 2020 ). A NASA-TLX score of 55/100 was expected which is based on the average score for the performance of cognitive tasks, physical activity and video gaming ( Grier, 2015 ).…”

Effects of an Animated Blood Clot Technology (Visual Clot) on the Decision-Making of Users Inexperienced in Viscoelastic Testing: Multicenter Trial

“…As secondary outcomes of this study, we assessed the diagnostic confidence binary as unconfident or confident, and the perceived workload using the raw NASA (National Aeronautics and Space Administration) Task Load Index questionnaire. This subjective workload assessment tool has been validated in many different areas, including health care [ 18 - 23 ]. The raw questionnaire defines the total perceived workload as the arithmetic mean of 6 different, workload-associated subscores [ 23 - 25 ].…”

“…This subjective workload assessment tool has been validated in many different areas, including health care [ 18 - 23 ]. The raw questionnaire defines the total perceived workload as the arithmetic mean of 6 different, workload-associated subscores [ 23 - 25 ]. This study did not investigate the physical demand subscore, as our tasks were not physically challenging.…”

Improving Visual-Patient-Avatar Design Prior to Its Clinical Release: A Mixed Qualitative and Quantitative Study

“…This type of display eliminates the need for users to translate the monitor’s numbers before interpreting them cognitively, and is better adapted to the strengths of human sensory perception and information processing [ 13 , 14 , 15 ]. In several computer-based studies, Visual-Patient-avatar facilitated time-efficient monitoring, increased the users’ diagnostic confidence and lowered their perceived workload [ 13 , 14 , 15 , 16 , 17 ]. In addition, Visual-Patient-avatar improved the ability to monitor multiple patients simultaneously [ 18 ], and was described as intuitive and easy to learn [ 19 , 20 , 21 ].…”