Cyberphysical Security Through Resiliency: A Systems-Centric Approach

Fleming, Craig; Elks, Carl; Bakirtzis, Georgios; Adams, Stephen; Carter, Bryan; Beling, Peter A.; Horowitz, Barry M.

doi:10.1109/mc.2020.3039491

Cited by 9 publications

(2 citation statements)

References 17 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Its general nature also makes it flexible and applicable across domains and system types, from aviation 2 to healthcare. 52 It has traditionally been used to support safety and security analysis, [49][50][51][53][54][55] however, and novel extensions and amendments may be needed address the novelties of AI, AIES, and the challenges of EA given DoD's specific EPs.…”

Section: Analyze Causal Factorsmentioning

confidence: 99%

On the role of loss-driven systems analysis in assessing AI ethics

Cody,

Freeman,

Beling

2024

Assurance and Security for AI-enabled Systems

View full text Add to dashboard Cite

As machine learning (ML) models are integrated more and more into critical systems, questions regarding their ethical use intensify. This paper advocates for a loss-driven engineering approach, incorporating concepts from systems-theoretic process analysis (STPA), to identify external systems, technologies, and processes essential for ethical ML deployment and therefore crticial to assessing AI ethics. STPA facilitates a deep analysis of potential hazards and system-level vulnerabilities, generating actionable insights for designing support systems and safeguards. Resilience engineering principles can be utilized to convert these insights into testable requirements for assessing AI ethics. This innovative, multi-disciplinary approach addresses a critical gap in current ML practices by extending ethical evaluation beyond the model, offering a robust framework for the responsible development and deployment of AI technologies.

show abstract

Section: Analyze Causal Factorsmentioning

confidence: 99%

On the role of loss-driven systems analysis in assessing AI ethics

Cody,

Freeman,

Beling

2024

Assurance and Security for AI-enabled Systems

View full text Add to dashboard Cite

show abstract

“…In the running scenario, early-phase testing could take the form of building and testing a gridworld that models the high-level decision-making for the uav. In this stage, the designer would identify anomalous behavior such as locations that create deadlocks, thereby enabling design decisions to mitigate situations that lead to task degradation [11].…”

Section: Dynamic Certificationmentioning

confidence: 99%

Dynamic Certification for Autonomous Systems

Bakirtzis¹,

Carr²,

Danks³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous systems are often deployed in complex sociotechnical environments, such as public roads, where they must behave safely and securely. Unlike many traditionally engineered systems, autonomous systems are expected to behave predictably in varying "open world" environmental contexts that cannot be fully specified formally. As a result, assurance about autonomous systems requires us to develop new certification methods and mathematical tools that can bound the uncertainty engendered by these diverse deployment scenarios, rather than relying on static tools.More specifically, autonomous systems increasingly use algorithms trained from data to predict and control behavior in previously unencountered contexts. The use of learning is a critical step to engineer autonomy that can successfully operate in heterogeneous contexts, including complex sociotechnical settings, but current certification methods are insufficient to address the dynamic, adaptive nature of learning.We propose the dynamic certification of autonomous systems-the iterative revision of permissible ⟨use, context⟩ pairs for a system-rather than prespecified tests that a system must pass to be certified. Dynamic certification offers the ability to "learn while certifying, " thereby opening additional opportunities to shape the development of an autonomous technology. This comprehensive, exploratory testing shaped by proposed deployment can enable iterative selection of appropriate contexts of use. More specifically, we propose dynamic certification and modeling involving three testing stages: early-phase testing, transitional testing, and confirmatory testing. Movement between testing stages is not unidirectional; we can shift in any direction depending on our current state of knowledge and intended deployments.We describe these stages in more detail below, but the key is that these stages enable system designers and regulators to both learn about and assure that engineered autonomous systems will operate within the bounds of acceptable risk.

show abstract