Extending Causal Models from Machines into Humans

Abstract

Causal Models are increasingly suggested as a means to reason about the behavior of cyber-physical systems in socio-technical contexts. They allow us to analyze courses of events and reason about possible alternatives. Until now, however, such reasoning is confined to the technical domain and limited to single systems or at most groups of systems. The humans that are an integral part of any such socio-technical system are usually ignored or dealt with by "expert judgment". We show how a technical causal model can be extended with models of human behavior to cover the complexity and interplay between humans and technical systems. This integrated socio-technical causal model can then be used to reason not only about actions and decisions taken by the machine, but also about those taken by humans interacting with the system. In this paper we demonstrate the feasibility of merging causal models about machines with causal models about humans and illustrate the usefulness of this approach with a highly automated vehicle example. Bottle Shatters Billy throws Suzy hits Billy hits Suzy throws (a) Classic version. Bottle Shatters Billy throws Suzy hits Billy hits Suzy throws Military Training Cond. Reaction(b) With military conditioning.Figure 1: The rock-throwing example.definitions of causality that will not lead to such counterintuitive results culminated in the Halpern-Pearl definition of causality [10]. It resolves the issues by introducing a so called preemption relation [12] that can express that Suzy's throw preempted Billy's throw (see Figure 1a). While such a model expresses the objective facts very well, we cannot reason about why Suzy threw faster. To do so, we would need a model of Suzy's mind. If, for example, she was a soldier and, as part of her training, was conditioned to throw rocks the moment a bottle appeared in her field of view 1 , we might not simply say that "Suzy throwing the rock caused the bottle to shatter", but extend our causal chain to "Her military training caused Suzy to automatically throw the rock at the bottle, shattering it". Instead of just blaming Suzy, we could also consider her military training.Returning to CPS and their interaction with humans, we can now utilize existing models of human behavior, as in Figure 1b, transform them to causal models and link them with causal models of the technical systems. Instead of just saying "The car crashed, because the driver pressed the red button", we can say that "Drivers are conditioned to press the red button in an emergency; this lead to the driver pressing the button and the car crashing". While it is true that we might not have enough data in many cases, in the cases where we do have enough data, like the example in Section 3, and can actually extend the causal model into the human mind, we can gain valuable insights and avoid the unsatisfying generic answer "human error".In this paper we investigate the problem of joining causal models of technical systems with causal models of their operators and people they interact with. As a solut...

Accountability for Cyber- Physical Systems

“…In [Kacianka and Pretschner, 2018, Kacianka et al, 2019a, Kacianka et al, 2019b we have shown that this works in principle.…”

“…Taken from [Kacianka et al, 2019a]. 4.3 HTA example given in [Kacianka et al, 2019b] that models a driver changing a lane. .…”

“…Computer systems are unique in the fact that we usually have models of different facets 8 of them available and can often use them to derive SCMs [Kacianka et al, 2019a]. These technical models can then be connected to their socio-technical context [Kacianka et al, 2019b, Kacianka and P öchhacker, 2020, providing holistic causal models that span system boundaries. Of course, the quality of any analysis depends on the accuracy of the models.…”

“…This thesis is built on ideas that were first published in several papers. [Kacianka et al, 2016] outlined our initial idea, [Kacianka et al, 2017] systematically mapped the literature on accountability, [Kacianka and Pretschner, 2018] outlined our initial thoughts on formalizing definitions of accountability, [Kacianka et al, 2019a] showed how to derive causal models from system models, [Kacianka et al, 2019b] showed how we can transform models of human behavior to causal models, [Kacianka and P öchhacker, 2020] discussed the social problems in constructing causal models and presented the core idea of this thesis, using structural causal models to express accountability definitions and compare them to system models. Chapter 5 is unpublished, but under review at the submission of this thesis.…”

From Checking to Inference: Actual Causality Computations as Optimization Problems

“…Similar to humans, it is useful for systems in investigating security protocols [21], safety accidents [22], software or hardware models [5,23,9], and database queries [28]. More importantly, actual causality is central for enabling social constructs such as accountability in Cyber-Physical systems [16,19,17], in information systems [10], and explainability in artificial intelligence systems [29].…”