Towards a Cognitive Theory of Cyber Deception

Abstract

This work is an initial step toward developing a cognitive theory of cyber deception. While widely studied, the psychology of deception has largely focused on physical cues of deception. Given that present-day communication among humans is largely electronic, we focus on the cyber domain where physical cues are unavailable and for which there is less psychological research. To improve cyber defense, researchers have used signaling theory to extended algorithms developed for the optimal allocation of limited defense resources by using deceptive signals to trick the human mind. However, the algorithms are designed to protect against adversaries that make perfectly rational decisions. In behavioral experiments using an abstract cybersecurity game (i.e., Insider Attack Game), we examined human decision-making when paired against the defense algorithm. We developed an instance-based learning (IBL) model of an attacker using the Adaptive Control of Thought-Rational (ACT-R) cognitive architecture to investigate how humans make decisions under deception in cyber-attack scenarios. Our results show that the defense algorithm is more effective at reducing the probability of attack and protecting assets when using deceptive signaling, compared to no signaling, but is less effective than predicted against a perfectly rational adversary. Also, the IBL model replicates human attack decisions accurately. The IBL model shows how human decisions arise from experience, and how memory retrieval dynamics can give rise to cognitive biases, such as confirmation bias. The implications of these findings are discussed in the perspective of informing theories of deception and designing more effective signaling schemes that consider human bounded rationality.

Joint human and autonomy teaming for defense: status, challenges, and perspectives

“…For instance, humans find it difficult to detect an attacker's intentions in the context of cybersecurity teams 89 due to human biases. 90 Human biases are often used by cybercriminals to conduct phishing attacks and obtain credentials to access an organization's systems, for example. 91 The use of AI biases can also be used by cybercriminals as a means of attack.…”

Many phish in the C$\mathcal{C}$: A coexisting‐choice‐criteria model of security behavior

“…Contextual cues also feature centrally in models of end‐user response to phishing emails (Cranford et al., 2019) based on instance‐based learning theory (IBLT) (Gonzalez et al., 2003). The latter, which is a model of dynamic decision making in cognitive science, also draws on instance‐based learning algorithms (IBLAs) (Aha et al., 1991) from machine learning.…”

“…As noted above, Equations () through () build on the conceptual apparatus introduced in the SDT‐based phishing susceptibility literature (Kaivanto, 2014). There are close parallels between the concepts employed here—histories of decision contexts $$\lbrace Z_i\rbrace _{<t}$$, decision‐attributes $$\lbrace {\bm{\alpha }}\rbrace _{<t}$$, decision‐outcomes $$\lbrace D_i\rbrace _{<t}$$, as well as match quality $$m_i^c(\bm{\alpha })$$—and those employed in IBLT (Cranford et al., 2019; Gonzalez et al., 2003). Hence, the machine learning and NLP techniques developed in IBLT can in principle be used in computation implementation of ().…”

Learning and Dynamic Decision Making

“…Importantly, phishing emails often mimic benign emails-meaning that decision makers, who are influenced by typical memory effects such as recency and frequency of past instances, are susceptible to the cognitive biases that emerge from these very processes (Singh et al, 2019). An IBL model was built to emulate end-used classification decisions of emails (i.e., as phishing or benign), and the results from this model were compared to the classification decisions of humans in an email-processing task (Cranford et al, 2021). The model has turned out to be very accurate at predicting human's classification decisions.…”