Enhanced cognitive workload evaluation in 3D immersive environments with TOPSIS model

Abstract

Research puts forward perception-based cognitive workload evaluation methods to help VR developers and users measuring their workload when playing with a VR application. Approaches to measure workload based on biosensors have progressed significantly, while evaluation based on subjective methods still rely on standard questionnaires such as the NASA-TLX table, the Subjective Workload Assessment Technique and the Modified Cooper Harper scale. The pre-defined questions enable operators to carry out experiments and analyse the data more easily than with biofeedback. However, the subjective evaluation process can bias the results because of unperceived internal changes and unknown factors among users. It is therefore necessary to have a method to handle and analyse this uncertainty. We propose to use the Technique for Order Performance by Similarity to Ideal Solution (TOPSIS) model to analyse the NASA-TLX table for measuring the overall user workload instead of using the classical weighted sum method. To show the advantage of the TOPSIS approach, we performed a user experiment to validate the approach and its application to VR, considering factors including the VR platform and the scenario density. Three different weighting methods, including the fuzzy Analytic Hierarchy Process (AHP) from fuzzy logic, the classical weighting based on pairwise comparison and the uniform weighting method, were tested to see the applicability of the TOPSIS model. The results from TOPSIS were consistent with those from other evaluation methods; a significant reduction in the coefficient of variation (CV) was observed when using the TOPSIS model to analyse the NASA-TLX scores, indicating an enhanced precision of the workload evaluation by the TOPSIS method. Our work has a potential application for VR designers and experimenters to compare cognitive workload among conditions and to optimize the settings.

Cognitive Matching of Design Subjects in Product Form Evolutionary Design

“…e determined alternative scheme should be located at the shortest geometric distance from the optimal scheme and the longest geometric distance from the worst scheme [30]. As a multicriteria decision analysis method, TOPSIS has been successfully applied to industrial technology [31,32], computer science [33][34][35], medical problems [36], applied statistics [37][38][39][40], environmental protection [41,42], and other fields.…”

Developing an Integrated Evaluation Model for Physician Comprehensive Workload Tethered to Outpatient Practice: An Empirical Study From China

“…Currently, the onset of modern technology and automation has greatly shifted the workload paradigm from the physical domain to the mental domain ( 16 , 17 ), and some researchers thereby used mental workload as the measure of an individual's workload ( 18 ). Mental workload is also a term referring to the cost of completing a work task, and it can be defined as the amount of brain or cognitive resources used/consumed per unit time to reach the acceptable performance required by the work task ( 19 ). Currently, mental workload is widely accepted to be defined as the portion of a human operator's limited capacity actually required to perform a specific task under the assumption that humans have a fixed amount of processing capacity; and the task inherently requires processing resources, and the more difficult the task, the higher the processing capacity required for acceptable objective and subjective performance ( 16 , 17 ).…”

“…For example, given that there are few instruments for measuring or diagnosing Chinese physicians' mental workload, our research team developed a mental workload scale verified with good reliability and validity based on the combination of dimensions of NASA-TLX and SWAT in 2018 ( 7 ). Although subjective measures of mental workload are with high face validity, ease of use, participant acceptability, low cost and known sensitivity to workload variation, there remain some methodological issues to be solved, such as trading off the intrusiveness of live ratings against the retrospective bias of post-task ratings ( 16 , 17 , 19 ). Accordingly, once a work task is finished, workload evaluation of the human operator is carried out immediately, possibly minimizing the recall bias.…”

“…However, the insensitivity of some of the measurements based on these performance-based techniques is one disadvantage, since there is not a simple linear, but an inverted U relationship between mental workload and job performance. That is, a work task with a low demanding mental workload could achieve an excellent performance during the beginning stage of the task, but then the performance will be degraded as the human operator becomes fatigued or distracted, possibly leading to confounded results ( 19 ). Accordingly, using task performance techniques with other mental workload assessment methods may improve the quality of the measurement ( 19 ).…”

“…That is, a work task with a low demanding mental workload could achieve an excellent performance during the beginning stage of the task, but then the performance will be degraded as the human operator becomes fatigued or distracted, possibly leading to confounded results ( 19 ). Accordingly, using task performance techniques with other mental workload assessment methods may improve the quality of the measurement ( 19 ).…”