Who Did This: Identifying Causes of Emergent Behavior in Complex Systems

Szabo, Claudia; Groot, J. de; McAtee, Thomas; Pyromallis, Christo

doi:10.1109/ssci44817.2019.9003143

Cited by 3 publications

(1 citation statement)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, metrics such as system entropy, complexity, and self-organization have been linked to desired emergent behavior in past work. 10,27,36 With the formulation of resilience as a fluctuation in available knowledge about other components, information theory approaches 68 could be used to further expand understanding.…”

Section: Discussionmentioning

confidence: 99%

A simulation and experimentation architecture for resilient cooperative multiagent reinforcement learning models operating in contested and dynamic environments

Honhaga,

Szabo

2024

SIMULATION

View full text Add to dashboard Cite

Cooperative multiagent reinforcement learning approaches are increasingly being used to make decisions in contested and dynamic environments, which tend to be wildly different from the environments used to train them. As such, there is a need for a more in-depth understanding of their resilience and robustness in conditions such as network partitions, node failures, or attacks. In this article, we propose a modeling and simulation framework that explores the resilience of four c-MARL models when faced with different types of attacks, and the impact that training with different perturbations has on the effectiveness of these attacks. We show that c-MARL approaches are highly vulnerable to perturbations of observation, action reward, and communication, showing more than 80% drop in the performance from the baseline. We also show that appropriate training with perturbations can dramatically improve performance in some cases, however, can also result in overfitting, making the models less resilient against other attacks. This is a first step toward a more in-depth understanding of the resilience c-MARL models and the effect that contested environments can have on their behavior and toward resilience of complex systems in general.

show abstract