Towards Mixed-Initiative Human–Robot Interaction: Assessment of Discriminative Physiological and Behavioral Features for Performance Prediction

Abstract

The design of human–robot interactions is a key challenge to optimize operational performance. A promising approach is to consider mixed-initiative interactions in which the tasks and authority of each human and artificial agents are dynamically defined according to their current abilities. An important issue for the implementation of mixed-initiative systems is to monitor human performance to dynamically drive task allocation between human and artificial agents (i.e., robots). We, therefore, designed an experimental scenario involving missions whereby participants had to cooperate with a robot to fight fires while facing hazards. Two levels of robot automation (manual vs. autonomous) were randomly manipulated to assess their impact on the participants’ performance across missions. Cardiac activity, eye-tracking, and participants’ actions on the user interface were collected. The participants performed differently to an extent that we could identify high and low score mission groups that also exhibited different behavioral, cardiac and ocular patterns. More specifically, our findings indicated that the higher level of automation could be beneficial to low-scoring participants but detrimental to high-scoring ones, and vice versa. In addition, inter-subject single-trial classification results showed that the studied behavioral and physiological features were relevant to predict mission performance. The highest average balanced accuracy (74%) was reached using the features extracted from all input devices. These results suggest that an adaptive HRI driving system, that would aim at maximizing performance, would be capable of analyzing such physiological and behavior markers online to further change the level of automation when it is relevant for the mission purpose.

Mixed-Initiative Human-Automated Agents Teaming: Towards a Flexible Cooperation Framework

“…The Figure 5 shows some ET and ECG features collected during experiments. More specifically, our results, recently published in [7], indicated that the higher level of automation could be beneficial to low-scoring participants but detrimental to high-scoring ones and vice versa. In addition, inter-subject single-trial classification results showed that the studied behavioral and physiological features were relevant to predict mission performance.…”

“…The firefighter robot game. A more recent project, based on a firefighter robotic mission [18,9,7] has been addressed in our lab in order to mitigate the drawbacks of our previous approaches: (i) the need for sufficient data to feed (MO)MDP models; (ii) the definition of a global common score reflecting mission performance, and being dependent of the efficiency of all involved agents; (iii) the implementation of a monitoring system to estimate the cognitive state of the human-operator during the interaction; and finally (iv) the evaluation in situ of the resulting (MO)MDP policy to drive such an interaction system.…”

“…One may note that as the ground tank level increases HRV decreases. It may relates to an increase of the cognitive load [8,7] of the human operator. works to encourage Internet users to carry out the mission in order to collect as much anonymous data as possible.…”

“…During these experiments several volunteers equipped with eye-tracking (ET) and electrocardiogram (ECG) have played the Firefighter Robot game following an experimental protocol including 4 mission re-alizations per participant. The acquired physiological and behavioral data were studied and allowed us to reveal interesting results concerning performance prediction [7].…”

“…The combination of the methods suggested in [9] and in [7] is under study. The aim will be to proceed to a MOMDP model learning (with all experimental and crowdsourcing data acquired) followed by the policy computation.…”