Understanding and Formalizing Accountability for Cyber-Physical Systems

Kacianka, Severin; Pretschner, Alexander

doi:10.1109/smc.2018.00536

Cited by 11 publications

(14 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Causality is an essential building block to answer complex questions of accountability [14]. It allows us to reason about different courses of actions and go beyond mere correlation based reasoning.…”

Section: Introductionmentioning

confidence: 99%

Extending Causal Models from Machines into Humans

Kacianka¹,

Ibrahim²,

Pretschner³

et al. 2019

Electron. Proc. Theor. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

Causal Models are increasingly suggested as a means to reason about the behavior of cyber-physical systems in socio-technical contexts. They allow us to analyze courses of events and reason about possible alternatives. Until now, however, such reasoning is confined to the technical domain and limited to single systems or at most groups of systems. The humans that are an integral part of any such socio-technical system are usually ignored or dealt with by "expert judgment". We show how a technical causal model can be extended with models of human behavior to cover the complexity and interplay between humans and technical systems. This integrated socio-technical causal model can then be used to reason not only about actions and decisions taken by the machine, but also about those taken by humans interacting with the system. In this paper we demonstrate the feasibility of merging causal models about machines with causal models about humans and illustrate the usefulness of this approach with a highly automated vehicle example. Bottle Shatters Billy throws Suzy hits Billy hits Suzy throws (a) Classic version. Bottle Shatters Billy throws Suzy hits Billy hits Suzy throws Military Training Cond. Reaction(b) With military conditioning.Figure 1: The rock-throwing example.definitions of causality that will not lead to such counterintuitive results culminated in the Halpern-Pearl definition of causality [10]. It resolves the issues by introducing a so called preemption relation [12] that can express that Suzy's throw preempted Billy's throw (see Figure 1a). While such a model expresses the objective facts very well, we cannot reason about why Suzy threw faster. To do so, we would need a model of Suzy's mind. If, for example, she was a soldier and, as part of her training, was conditioned to throw rocks the moment a bottle appeared in her field of view 1 , we might not simply say that "Suzy throwing the rock caused the bottle to shatter", but extend our causal chain to "Her military training caused Suzy to automatically throw the rock at the bottle, shattering it". Instead of just blaming Suzy, we could also consider her military training.Returning to CPS and their interaction with humans, we can now utilize existing models of human behavior, as in Figure 1b, transform them to causal models and link them with causal models of the technical systems. Instead of just saying "The car crashed, because the driver pressed the red button", we can say that "Drivers are conditioned to press the red button in an emergency; this lead to the driver pressing the button and the car crashing". While it is true that we might not have enough data in many cases, in the cases where we do have enough data, like the example in Section 3, and can actually extend the causal model into the human mind, we can gain valuable insights and avoid the unsatisfying generic answer "human error".In this paper we investigate the problem of joining causal models of technical systems with causal models of their operators and people they interact with. As a solut...

show abstract

Section: Introductionmentioning

confidence: 99%

Extending Causal Models from Machines into Humans

Kacianka¹,

Ibrahim²,

Pretschner³

et al. 2019

Electron. Proc. Theor. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…To illustrate our point, we will expand on the example given by Kacianka and Pretschner (2018) . They examined a prominently reported accident involving an autonomous vehicle operated by Uber in Arizona ( Elish, 2019 ).…”

Section: From Causality To Accountability: the Computer Science Appromentioning

confidence: 99%

“…As a result, causality seems to be a promising approach to tackle issues of algorithmic accountability. This led computer science scholars to explore the formal expression of algorithmic accountability as a structural causal model (SCM) ( Kacianka and Pretschner, 2018 ). The underlying idea is that accountability always requires causality ( Kacianka et al, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

Algorithmic Accountability in Context. Socio-Technical Perspectives on Structural Causal Models

Poechhacker

Kacianka

2021

Front. Big Data

Self Cite

View full text Add to dashboard Cite

The increasing use of automated decision making (ADM) and machine learning sparked an ongoing discussion about algorithmic accountability. Within computer science, a new form of producing accountability has been discussed recently: causality as an expression of algorithmic accountability, formalized using structural causal models (SCMs). However, causality itself is a concept that needs further exploration. Therefore, in this contribution we confront ideas of SCMs with insights from social theory, more explicitly pragmatism, and argue that formal expressions of causality must always be seen in the context of the social system in which they are applied. This results in the formulation of further research questions and directions.

show abstract

“…A potential approach to tackling trust's challenge towards minimizing vulnerabilities is based on accountability [10]. Accountability enables "the uncovering of causes for events and helps understand who or what is responsible for these events" [11]. By increasing this understanding, accountability can increase trust and thus improve situations were an IBP cannot be based on a long lasting collaboration.…”

Section: Introductionmentioning

confidence: 99%

“…e.g. [11][12][13][14]). Given this conceptual shortcoming, this paper addresses the research question: What is the scope of accountability facilitating to substantiate trust in the context of IBP?…”

Section: Introductionmentioning

confidence: 99%

Dimensions of Accountability in Inter-organizational Business Processes

Buchinger¹,

Kuhn²,

Balta³

2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Inter-organizational business processes are the basis of a globalized, highly dynamic, and digitalized world, enabling faster and cost-effective transactions. At the same time, they raise business vulnerabilities. A particular vulnerability is linked to the substantiation of trust between actors in dynamic business relationships, as trust affects interdependencies and complexity. An approach to address this vulnerability is the introduction of accountability mechanisms. Extant research suggests that accountability enables revealing causality and a transparent allocation of responsibilities for each process step. Thereby, corresponding actors can judge upon misbehavior and verify trust claims. Unfortunately, a thorough understanding of accountability and its dimensions accountability in the context of IBP is still missing. To address this gap, we develop a framework with dimensions of accountability. We demonstrate the resulting framework in an industrial supply chain case and derive implications for theory and practice.

show abstract

Understanding and Formalizing Accountability for Cyber-Physical Systems

Cited by 11 publications

References 13 publications

Extending Causal Models from Machines into Humans

Extending Causal Models from Machines into Humans

Algorithmic Accountability in Context. Socio-Technical Perspectives on Structural Causal Models

Dimensions of Accountability in Inter-organizational Business Processes

Contact Info

Product

Resources

About