A feedforward neural network for drone accident prediction from physiological signals

Abstract

PurposeAs drones are rapidly transforming tasks such as mapping and surveying, safety inspection and progress monitoring, human operators continue to play a critical role in ensuring safe drone missions in compliance with safety regulations and standard operating procedures. Research shows that operator's stress and fatigue are leading causes of drone accidents. Building upon the authors’ past work, this study presents a systematic approach to predicting impending drone accidents using data that capture the drone operator's physiological state preceding the accident.Design/methodology/approachThe authors collect physiological data from 25 participants in real-world and virtual reality flight experiments to design a feedforward neural network (FNN) with back propagation. Four time series signals, namely electrodermal activity (EDA), skin temperature (ST), electrocardiogram (ECG) and heart rate (HR), are selected, filtered for noise and used to extract 92 time- and frequency-domain features. The FNN is trained with data from a window of length t = 3…8 s to predict accidents in the next p = 3…8 s.FindingsAnalysis of model performance in all 36 combinations of analysis window (t) and prediction horizon (p) combinations reveals that the FNN trained with 8 s of physiological signal (i.e. t = 8) to predict drone accidents in the next 6 s (i.e. p = 6) achieved the highest F1-score of 0.81 and AP of 0.71 after feature selection and data balancing.Originality/valueThe safety and integrity of collaborative human–machine systems (e.g. remotely operated drones) rely on not only the attributes of the human operator or the machinery but also how one perceives the other and adopts to the evolving nature of the operational environment. This study is a first systematic attempt at objective prediction of potential drone accident events from operator's physiological data in (near-) real time. Findings will lay the foundation for creating automated intervention systems for drone operations, ultimately leading to safer jobsites.

The effect of mental schema evolution on mental workload measurement: an EEG study with simulated quadrotor UAV operation

“…However, current unmanned aerial vehicle (UAV) systems are complex to operate safely, and operators must undergo rigorous training and testing before being granted flying licenses. The Federal Aviation Administration predicts that the number of UAVs will triple by 2023 (Sakib et al 2021a), and it is expected that the increased use of UAVs will lead to more UAV-related accidents (Minucci et al 2020). To perform tasks efficiently, an operator must maintain constant situational awareness, and changes in the mental workload may lead to mental stress and mental fatigue for the operator (Sakib et al 2021b), which could deteriorate performance and ultimately endanger people and property (Nouacer et al 2020).…”

Integrating Biofeedback and Artificial Intelligence into eXtended Reality Training Scenarios: A Systematic Literature Review

“…Information always travels in one direction-from the input layer to the output layer-and never goes backward. ++ (Sakib et al, 2022), (Suhaimi et al, 2021) (continued) A gradient boost machine is a type of machine learning technique that uses an ensemble of weak prediction models to perform regression and classification tasks. ++ (Abujelala et al, 2021), (Bin Suhaimi et al, 2020), (Granato et al, 2020), (Martin et al, 2020), (Recenti et al, 2021), (Suhaimi, et al, 2022) K-Nearest Neighbour (KNN)…”

Unmanned aerial vehicles in the construction industry - Towards a protocol for safe preparation and flight of drones

“…In addition, UAV on a construction site can be distracting for workers, and this can worsen their overall safety performance and increase the number of accidents at work that do not directly involve a drone (Namian et al, 2021). Previous studies also show that stress and fatigue of the user/ drone pilot are the main causes of accidents involving these devices (Sakib et al, 2021).…”