SAHRTA: A Supervisory-Based Adaptive Human-Robot Teaming Architecture

Predicting task performance for intelligent human-machine interactions

“…A relationship exists between workload and overall task performance (Wickens et al, 2004 ); thus, workload information may be used to predict future performance. This information was obtained via a workload assessment algorithm (Heard and Adams, 2019 ; Heard et al, 2019 ) that estimated the overall workload and each workload component (cognitive, physical, speech, visual, and auditory) every 5 s. This frequency was chosen in order to balance system adaptation rates and workload estimation accuracy (Heard et al, 2020 ). The algorithm relied on physiological (e.g., heart-rate) data and a 5-layer neural network structure and has been validated in multiple human-machine teaming paradigms (Heard et al, 2019 ).…”

“…Additionally, the 5-min time-frame per condition reflects the time that physiological signals need to identify a workload transition (Reimer et al, 2009 ). Multiple works have investigated physiological responses to various workload states (Castor, 2003 ; Cain, 2007 ; Heard, 2019 ). For example, there is typically an increase in heart-rate when transitioning from a lower workload state to a higher workload state.…”

“…The simulation, depicted in Figure 1 , incorporated four concurrent tasks: tracking, system management, resource management, and communication monitoring. Each task elicits different demand levels (e.g., cognitive, visual, physical), which was captured in the IMPRINT Pro software (Heard, 2019 ). Participants also have differing skill levels, where some tasks may be easier than others (e.g., the tracking task may be easier for someone with prior flight simulator experience).…”

“…Conflicts can occur between the workload channels (i.e., between auditory and speech) and reduce task performance. More intelligent adaptation strategies can be implemented if the system understands the complete workload state (Heard et al, 2020 ). For example, if a human's speech workload is overloaded, then an audible alert creates a channel conflict and potentially reduces performance.…”

Real-Time Speech Workload Estimation for Intelligent Human- Machine Systems

“…The speech workload algorithm was used to estimate workload in real-time during an adaptive human-machine system pilot evaluation (10 participants using the described foundational experimental design) (Heard, Fortune, and Adams, 2020) with a window size of 5s. There is no ground truth model to compare the speech-workload estimates against, due to the real-time and adaptive nature of the evaluation.…”