Virtual Workload Measurement for Assessing Systems Utilizing Automation Technology

Abstract

With the increasing integration of automation technologies, the role of the operator is changing from sole actor to a shared supervisor/actor role. Studies on unmanned ground vehicle operators and recent crashes partially blamed on automation technologies demonstrate the need to measure and assess operator awareness and workload. Overcoming these challenges requires an assessment early in the design cycle for operator awareness and workload. This methodology integrates concepts from cognitive engineering into operations analysis to better capture and analyze the effectiveness of increasingly automated systems. An agent-based model is created using Operational Event Sequence Diagrams and concepts from situation awareness research to guide agent formulation. The agent rule set is then mapped to the NASA Task Load Index scales to provide a dynamic output throughout the simulation. A traffic model is built in AFSIM to compare the mental workload associated with city versus highway driving. The dynamic workload measurement is the first step in a framework which will enable automation technologies to be traded during the conceptual design phase.

Cognitive Analyses for Interface Design Using Dual N-Back Tasks for Mental Workload (MWL) Evaluation

“…The performance average is shown in the percentage of correct responses for all participants (Table 4). The results found for overall NASA-TLX scores were in the corresponding ranges of Low (0-29), Medium (30)(31)(32)(33)(34)(35)(36)(37)(38)(39)(40)(41)(42)(43)(44)(45)(46)(47)(48)(49), and High (50-100), values stated according to the data collected [54] and very similar to the values expressed in other studies [53]. These results show the highest average values of NASA's dimensions in the highest mental workload level, except for the physical demand.…”

“…These classifications are then merged to obtain the task load index. The other part of the test proposes establishing an individual weighting for these subscales by allowing subjects to match them in pairs based on their perceived importance [42]. NASA-TLX and HTA have been used together in some studies in different domains, including evaluating mental workload in human system design for manufacturing operations [33,43], human performance in various tasks [40,42,44], and assessment of the subjective MWL with the NASA-TLX and N-Back tasks.…”

“…The other part of the test proposes establishing an individual weighting for these subscales by allowing subjects to match them in pairs based on their perceived importance [42]. NASA-TLX and HTA have been used together in some studies in different domains, including evaluating mental workload in human system design for manufacturing operations [33,43], human performance in various tasks [40,42,44], and assessment of the subjective MWL with the NASA-TLX and N-Back tasks. Researchers in a study obtained higher mental demand in a 3-Back task with average scores for NASA in a range of (38.68 ± 3.82), compared to an oddball condition with average scores for NASA in a range of (15.07 ± 1.78); these results were verified via an ocular aberration measurement as a physiological, mental workload measurement [45].…”