An improved understanding of how the brain allocates mental resources as a function of task difficulty is critical for enhancing human performance. Functional near infrared spectroscopy (fNIRS) is a field-deployable optical brain monitoring technology that provides a direct measure of cerebral blood flow in response to cognitive activity. We found that fNIRS was sensitive to variations in task difficulty in both real-life (flight simulator) and laboratory settings (tests measuring executive functions), showing increased concentration of oxygenated hemoglobin (HbO2) and decreased concentration of deoxygenated hemoglobin (HHb) in the prefrontal cortex as the tasks became more complex. Intensity of prefrontal activation (HbO2 concentration) was not clearly correlated to task performance. Rather, activation intensity shed insight on the level of mental effort, i.e., how hard an individual was working to accomplish a task. When combined with performance, fNIRS provided an estimate of the participants’ neural efficiency, and this efficiency was consistent across levels of difficulty of the same task. Overall, our data support the suitability of fNIRS to assess the mental effort related to human operations and represents a promising tool for the measurement of neural efficiency in other contexts such as training programs or the clinical setting.

In our anxiogenic and stressful world, the maintenance of an optimal cognitive performance is a constant challenge. It is particularly true in complex working environments (e.g. flight deck, air traffic control tower), where individuals have sometimes to cope with a high mental workload and stressful situations. Several models (i.e. processing efficiency theory, cognitive-energetical framework) have attempted to provide a conceptual basis on how human performance is modulated by high workload and stress/anxiety. These models predict that stress can reduce human cognitive efficiency, even in the absence of a visible impact on the task performance. Performance may be protected under stress thanks to compensatory effort, but only at the expense of a cognitive cost. Yet, the psychophysiological cost of this regulation remains unclear. We designed two experiments involving pupil diameter, cardiovascular and prefrontal oxygenation measurements. Participants performed the Toulouse N-back Task that intensively engaged both working memory and mental calculation processes under the threat (or not) of unpredictable aversive sounds. The results revealed that higher task difficulty (higher n level) degraded the performance and induced an increased tonic pupil diameter, heart rate and activity in the lateral prefrontal cortex, and a decreased phasic pupil response and heart rate variability. Importantly, the condition of stress did not impact the performance, but at the expense of a psychophysiological cost as demonstrated by lower phasic pupil response, and greater heart rate and prefrontal activity. Prefrontal cortex seems to be a central region for mitigating the influence of stress because it subserves crucial functions (e.g. inhibition, working memory) that can promote the engagement of coping strategies. Overall, findings confirmed the psychophysiological cost of both mental effort and stress. Stress likely triggered increased motivation and the recruitment of additional cognitive resources that minimize its aversive effects on task performance (effectiveness), but these compensatory efforts consumed resources that caused a loss of cognitive efficiency (ratio between performance effectiveness and mental effort).

Objective: The aim of this study was to test whether inattentional deafness to critical alarms would be observed in a simulated cockpit. Background: The inability of pilots to detect unexpected changes in their auditory environment (e.g., alarms) is a major safety problem in aeronautics. In aviation, the lack of response to alarms is usually not attributed to attentional limitations, but rather to pilots choosing to ignore such warnings due to decision biases, hearing issues, or conscious risk taking. Method: Twenty-eight general aviation pilots performed two landings in a flight simulator. In one scenario an auditory alert was triggered alone, whereas in the other the auditory alert occurred while the pilots dealt with a critical windshear. Results: In the windshear scenario, 11 pilots (39.3%) did not report or react appropriately to the alarm whereas all the pilots perceived the auditory warning in the no-windshear scenario. Also, of those pilots who were first exposed to the no-windshear scenario and detected the alarm, only three suffered from inattentional deafness in the subsequent windshear scenario. Conclusion: These findings establish inattentional deafness as a cognitive phenomenon that is critical for air safety. Pre-exposure to a critical event triggering an auditory alarm can enhance alarm detection when a similar event is encountered subsequently. Application: Case-based learning is a solution to mitigate auditory alarm misperception.

An analysis of airplane accidents reveals that pilots sometimes purely fail to react to critical auditory alerts. This inability of an auditory stimulus to reach consciousness has been coined under the term of inattentional deafness. Recent data from literature tends to show that tasks involving high cognitive load consume most of the attentional capacities, leaving little or none remaining for processing any unexpected information. In addition, there is a growing body of evidence for a shared attentional capacity between vision and hearing. In this context, the abundant information in modern cockpits is likely to produce inattentional deafness. We investigated this hypothesis by combining electroencephalographic (EEG) measurements with an ecological aviation task performed under contextual variation of the cognitive load (high or low), including an alarm detection task. Two different audio tones were played: standard tones and deviant tones. Participants were instructed to ignore standard tones and to report deviant tones using a response pad. More than 31% of the deviant tones were not detected in the high load condition. Analysis of the EEG measurements showed a drastic diminution of the auditory P300 amplitude concomitant with this behavioral effect, whereas the N100 component was not affected. We suggest that these behavioral and electrophysiological results provide new insights on explaining the trend of pilots’ failure to react to critical auditory information. Relevant applications concern prevention of alarms omission, mental workload measurements and enhanced warning designs.

In the context of task sharing between a robot companion and its human partners, the notions of safe and compliant hardware are not enough. It is necessary to guarantee ergonomic robot motions. Therefore, we have developed Human Aware Manipulation Planner (Sisbot et al., 2010), a motion planner specifically designed for human-robot object transfer by explicitly taking into account the legibility, the safety and the physical comfort of robot motions. The main objective of this research was to define precise subjective metrics to assess our planner when a human interacts with a robot in an object hand-over task. A second objective was to obtain quantitative data to evaluate the effect of this interaction. Given the short duration, the "relative ease" of the object hand-over task and its qualitative component, classical behavioral measures based on accuracy or reaction time were unsuitable to compare our gestures. In this perspective, we selected three measurements based on the galvanic skin conductance response, the deltoid muscle activity and the ocular activity. To test our assumptions and validate our planner, an experimental set-up involving Jido, a mobile manipulator robot, and a seated human was proposed. For the purpose of the experiment, we have defined three motions that combine different levels of legibility, safety and physical comfort values. After each robot gesture the participants were asked to rate them on a three dimensional subjective scale. It has appeared that the subjective data were in favor of our reference motion. Eventually the three motions elicited different physiological and ocular responses that could be used to partially discriminate them.

Pupil diameter is a widely-studied cognitive load measure, which, despite its convenience for non-intrusive operator state monitoring in complex environments, is still not available for in situ measurements because of numerous methodological limitations. The most important of these limitations is the influence of pupillary light reflex. Hence, there is the need of providing a pupil-based cognitive load measure that is independent of light conditions. In this paper, we present a promising technique of pupillary signal analysis resulting in luminance-independent measure of mental effort that could be used in real-time without a priori on luminous conditions. Twenty-two participants performed a short-term memory task under different screen luminance conditions. Our results showed that the amplitude of pupillary dilation due to load on memory was luminance-dependent with higher amplitude corresponding to lower-luminance condition. Furthermore, our experimentation showed that load on memory and luminance factors express themselves differently according to frequency. Therefore, as our statistical analysis revealed, the ratio between low (0-1.6 Hz) and high frequency (1.6-4 Hz) bands (LF/HF ratio) of power spectral densities of pupillary signal is sensitive to the cognitive load but not to luminance. Our results are promising for the measurement of load on memory in ecological settings.

The review of literature in sociology and distributed artificial intelligence reveals that the occurrence of conflict is a remarkable precursor to the disruption of multi-agent systems. The study of this concept could be applied to human factors concerns, as man-system conflict appears to provoke perseveration behavior and to degrade attentional abilities with a trend to excessive focus. Once entangled in such conflicts, the human operator will do anything to succeed in his current goal even if it jeopardizes the mission. In order to confirm these findings, an experimental setup, composed of a real unmanned ground vehicle, a ground station is developed. A scenario involving an authority conflict between the participants and the robot is proposed. Analysis of the effects of the conflict on the participants' cognition and arousal is assessed through heart-rate measurement (reflecting stress level) and eye-tracking techniques (index of attentional focus). Our results clearly show that the occurrence of the conflict leads to perseveration behavior and can induce higher heart rate as well as excessive attentional focus. These results are discussed in terms of task commitment issues and increased arousal. Moreover, our results suggest that individual differences may predict susceptibility to perseveration behavior.

Objective: Examination of the performance and visual scanning of aircrews during final approach and an unexpected go-around maneuver. Background: Accident and incident analyses have revealed that goaround procedures are often imperfectly performed because of their complexity, their high time stress, and their rarity of occurrence that avails little time for practice. We wished to examine this experimentally and establish the frequency and nature of errors in both flight-performance and visual scanning. Method: We collected flight-performance (e.g., errors in procedures, excessive flight deviations) and eye-tracking data of 12 flight crews who performed final approach and go-around flight phases in realistic full-flight transport-category simulators. Results: The pilot performance results showed that two thirds of the crews committed errors including critical trajectory deviations during go-arounds, a precursor of accidents. Eye-tracking analyses revealed that the cross-checking process was not always efficient in detecting flight-path deviations when they occurred. Ocular data also highlighted different visual strategies between the 2 crew members during the 2 flight phases. Conclusion: This study reveals that the go-around is a challenging maneuver. It demonstrates the advantages of eye tracking and suggests that it is a valuable tool for the explicit training of attention allocation during go-arounds to enhance flight safety. In commercial aviation, the final-approach phase is one of the most dynamic flight segments, for which safety margins are minimum and time pressure is maximum. Numerous undesired external events or crew failure to stabilize the flight path or to configure the aircraft for landing could lead to the execution of a go-around. Flight crews, composed of a pilot flying (PF) and a pilot monitoring (PM), are trained to land the aircraft or to perform a go-around according to standard operating procedures (U.S. Department of Transportation, 2015). During go-around, the PF's task is to manage the missed-approach flight path while giving instructions to the PM (e.g., retracting the flaps and the landing gear). The PM carefully monitors the flight path and the status of the aircraft and executes the PF's instructions.