Gaps in the Control of Automated Vehicles on Roads

Abstract

Increased on-road testing and market availability of partially automated vehicles (AV) offers researchers and developers the opportunity to evaluate the AV's performance. The occurrence of new types of accidents involving AV's has sparked questions in regard to who is actually in control over and responsible for AV control. In this contribution, we suggest a potential discrepancy in AV control with the review of recently documented accidents involving AV's. The identification of a gap in control is performed using a recently formulated moral philosophical framework of Meaningful Human Control (MHC). This shows a discrepancy between the attribution of responsibility and the ability of a human to fulfil the role assigned to them. While a gap in control is not evident from the viewpoint of operational control, it requires the more intricate concept of MHC to expose it. Recommendations are further made that AV developers and vehicle approval authorities should consider control from a MHC perspective to avoid future gaps in control with the resulting consequences.

A human centric framework for the analysis of automated driving systems based on meaningful human control

“…It is with regard to partial automation, in which drivers are deemed to maintain levels of control from a viewpoint of monitoring, that there may be concerns over real meaningful control in practice. This has previously been identified and discussed in other works (Calvert et al 2020) and will be carried forward in consideration of the operationalisation of MHC.…”

“…Pacaux-Lemoine (2015) describes know-how (to operate) and know-how-to-cooperate as the ability to control a part of a process and an agent's ability to cooperate with other agents concerned by the process control. But knowing how, does not guarantee the ability to actually exert that control, which has been strongly argued in regard to automated driving by many from within the human factors and behavioural domains (Calvert et al 2020). By stating that the control should be meaningful, MHC explicitly includes an agent's ability and moral involvement to understand and operate, which can be clearly seen from the two conditions: traceability and trackability.…”

Four Responsibility Gaps with Artificial Intelligence: Why they Matter and How to Address them

“…They mostly consist of sets of normative requirements to promote a legally, ethically, and societally acceptable form of human control. Originally proposed in the context of lethal autonomous weapon systems (Amoroso & Tamburrini, 2019;Article36, 2014;Chengeta, 2016;Ekelhof, 2019;Horowitz & Scharre, 2015;Moyes, 2016;Schwarz, 2018), MHC has been recently investigated in the field of automated driving systems (Calvert et al, 2018(Calvert et al, , 2019(Calvert et al, , 2020Heikoop et al, 2019;Santoni de Sio & van den Hoven, 2018) and medical automation (Braun et al, 2020;Ficuciello et al, 2019). Relatedly, the importance as well as the difficulty of defining the kind of "meaningful human involvement" required to avoid responsibility gaps in automated decision-making is highlighted in the art.…”

A Conceptual Control System Description of Cooperative and Automated Driving in Mixed Urban Traffic With Meaningful Human Control for Design and Evaluation

“…traffic lights). There are also questions about gaps in control of CAVs [2,3] and moral choices by CAVs [4][5][6] that are present for highway traffic, and these are all more potent and amplified in the urban traffic environment [7]. With certain aspects of CAV control being questionable from a purely operational and human acceptance perspective [8,9], the concept of Meaningful Human Control (MHC) has been adopted for Cooperative and Automated Driving (CAD) to explicitly include human moral reasoning and ethical acceptability.…”