A survey of workload assessment algorithms

Heard, Jamison; Harriott, Caroline E.; Adams, Julie A.

doi:10.1109/thms.2017.2782483

Cited by 89 publications

(79 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cognitive load can be measured through subjective selfassessment metrics such as questionnaires; and objective metrics such as body motion, heart rate variability, and measures of pupil dilation over time derived from pupillometry [37], [38]. Pupillometry has recently gained popularity in applied psychology as a reliable proxy for cognitive load [39].…”

Section: Operator Cognitive Loadmentioning

confidence: 99%

Planetary Exploration With Robot Teams: Implementing Higher Autonomy With Swarm Intelligence

St-Onge

Kaufmann

Panerati

et al. 2020

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

Since the beginning of space exploration, Mars and the Moon have been explored with orbiters, landers, and rovers. Over forty missions have targeted Mars, and more than a hundred, the Moon. Developing novel strategies and technologies for exploring celestial bodies continues to be a focus of space agencies. Multi-robot systems are particularly promising for planetary exploration, as they are more robust to individual failure and have the potential to explore larger areas; however, there are limits to how many robots an operator can individually control. We recently took part in the European Space Agency's interdisciplinary equipment test campaign (PANGAEA-X) at a Lunar/Mars analogue site in Lanzarote, Spain. We used a heterogeneous fleet of Unmanned Aerial Vehicles (UAVs)-a swarm-to study the interplay of systems operations and human factors. Human operators directed the swarm via ad-hoc networks and data sharing protocols to explore unknown areas under two control modes: one in which the operator instructed each robot separately; and the other in which the operator provided general guidance to the swarm, which self-organized via a combination of distributed decision-making, and consensus building. We assessed cognitive load via pupillometry for each condition, and perceived task demand and intuitiveness via self-report. Our results show that implementing higher autonomy with swarm intelligence can reduce workload, freeing the operator for other tasks such as overseeing strategy, and communication. Future work will further leverage advances in swarm intelligence for exploration missions.

show abstract

Section: Operator Cognitive Loadmentioning

confidence: 99%

Planetary Exploration With Robot Teams: Implementing Higher Autonomy With Swarm Intelligence

St-Onge

Kaufmann

Panerati

et al. 2020

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

show abstract

“…For example, the neural net of a future automated driving system may be able to predict likelihood of driver error from inputs such as physiological sensors, vehicle control responses and analysis of ambient distractors without ever having to compute workload explicitly. Existing studies of algorithms that aggregate data from multiple sources to identify workload have utilized a range of machine learning classifiers including artificial neural networks, linear regression, linear discriminant analysis and support vector machines (Heard, Harriott, and Adams 2018). Algorithms can also be personalized to reflect individual variation in the responses most sensitive to workload, in effect assessing workload on a within-rather than a between-subjects basis (Teo et al 2018).…”

Section: Future Research Directionsmentioning

confidence: 99%

“…Current algorithms have various shortcomings described by Heard et al (2018) including limited generalizability, limited sampling of workload components and lack of verification in practical settings. They may also be difficult to interpret in relation to extant psychological and neuroscience theory.…”

Section: Future Research Directionsmentioning

confidence: 99%

What do subjective workload scales really measure? Operational and representational solutions to divergence of workload measures

Matthews

Winter

Hancock

2019

Theoretical Issues in Ergonomics Science

View full text Add to dashboard Cite

We examine the continuing use of subjective workload responses to index an operator's state, either by themselves or as part of a collective suite of measurements. Lack of convergence of subjective scales with physiological and performance-based measures calls into question whether there is any unitary workload construct that underpins conscious experience, physiological state and the individual's profile of task-related performance. We examine philosophical and measurement perspectives on the divergence problem, and we consider three possible solutions. First, difficulties in reliable and valid measurement of workload may contribute to divergence but do not fully explain it. Second, workload may be treated operationally: use of specific measures is justified by demonstrating their pragmatic utility in predicting important outcomes. Third, further efforts may be made to develop representational workload measurements that correspond to real empirical phenomena. Application of formal standards for test validity can identify multiple latent constructs supporting subjective workload, including those defining self-regulation in performance contexts. Physiological and performance-based assessments may define additional, distinct constructs. A resolution of the diversity issue is crucial for ergonomics since the invalid application of workload measurement will threaten exposed operators as well as many others who are served by the complex technological systems they control. Relevance to human factors/Relevance to ergonomics theory Workload assessment is important for multiple applications of ergonomics, but subjective and objective workload measures often diverge. Current ergonomics theory fails to explain this divergence. This article provides two possible solutions to this challenge. First, specific workload measures may be treated as operational variables that predict important real-world outcomes. This approach eschews theory in favor of pragmatic utility. It is compatible with emerging "big data" approaches to performance prediction. Second, subjective measures ARTICLE HISTORY

show abstract

“…The employment of secondary cognitive task (e.g. the n-back) during real activities does not fit the realistic conditions and may increase the actual workload level [21]. Moreover, because of the high individual variability of physiological responses, traditional statistical tests are not able to discover the relationship between cause and effect, so it is necessary to employ techniques that allow to take into account the individual characteristics to correctly define the level of workload, such the machine learning techniques [22].…”

Section: Machine Learning To Get Back Out-of-the-labmentioning

confidence: 99%

On the Use of Machine Learning for EEG-Based Workload Assessment: Algorithms Comparison in a Realistic Task

Sciaraffa

Aricò

Borghini

et al. 2019

Communications in Computer and Information Science

View full text Add to dashboard Cite

The measurement of the mental workload during real tasks by means of neurophysiological signals is still challenging. The employment of Machine Learning techniques has allowed a step forward in this direction, however, most of the work has dealt with binary classification. This study proposed to examine the surveys already performed in the context of EEG-based workload classification and to test different machine learning algorithms on real multitasking activity like the Air Traffic Management. The results obtained on 35 professional Air Traffic Controllers showed that a KNN algorithm allows discriminating up to three workload levels (low, medium and high) with more than 84 % of accuracy on average. Moreover, in such realistic employment it emerges how important is to opportunely choose the set of features to ward off that task-related confounds could affect the workload assessment.

show abstract

A survey of workload assessment algorithms

Cited by 89 publications

References 65 publications

Planetary Exploration With Robot Teams: Implementing Higher Autonomy With Swarm Intelligence

Planetary Exploration With Robot Teams: Implementing Higher Autonomy With Swarm Intelligence

What do subjective workload scales really measure? Operational and representational solutions to divergence of workload measures

On the Use of Machine Learning for EEG-Based Workload Assessment: Algorithms Comparison in a Realistic Task

Contact Info

Product

Resources

About