The computational relationship between reinforcement learning, social inference, and paranoia

Abstract

Theoretical accounts suggest heightened uncertainty about the state of the world underpin aberrant belief updates, which in turn increase the risk of developing a persecutory delusion. However, this raises the question as to how an agent’s uncertainty may relate to the precise phenomenology of paranoia, as opposed to other qualitatively different forms of belief. We tested whether the same population (n = 693) responded similarly to non-social and social contingency changes in a probabilistic reversal learning task and a modified repeated reversal Dictator game, and the impact of paranoia on both. We fitted computational models that included closely related parameters that quantified the rigidity across contingency reversals and the uncertainty about the environment/partner. Consistent with prior work we show that paranoia was associated with uncertainty around a partner’s behavioural policy and rigidity in harmful intent attributions in the social task. In the non-social task we found that pre-existing paranoia was associated with larger decision temperatures and commitment to suboptimal cards. We show relationships between decision temperature in the non-social task and priors over harmful intent attributions and uncertainty over beliefs about partners in the social task. Our results converge across both classes of model, suggesting paranoia is associated with a general uncertainty over the state of the world (and agents within it) that takes longer to resolve, although we demonstrate that this uncertainty is expressed asymmetrically in social contexts. Our model and data allow the representation of sociocognitive mechanisms that explain persecutory delusions and provide testable, phenomenologically relevant predictions for causal experiments.

D2/D3 dopamine supports the precision of mental state inferences and self-relevance of joint social outcomes

“…We note that previous work showed a Bayesian instantiation of this attributional model outperformed associative model variants [31]. Model 1 allowed separate uncertainties and likelihood weights for each attribution, identical to our prior work [31]; this model demonstrated that trait paranoia increased belief rigidity and self-other inconsistency, and by extension, may serve as a useful assay to test the mechanisms of haloperidol which is theorised to reduced paranoia. In line with general theories of belief updating [37], Model 2 hypothesised that beliefs would be updating with the same likelihood weight.…”

“…Current models include mechanistic explanations for perceived changes in the harmful intent and self-interest that might motivate the actions of another. Prior work suggests high trait paranoia is associated with rigid priors about the harmful intent of partners, and a belief that a partner’s actions are not consistent with their true intentions (Barnby et al, 2020b; 2022a). Several predictions can be made concerning the influence of dopamine D2/D3 antagonism on paranoia.…”

“…Following from our preregistered behavioural experiment (Barnby et al, 2020a), we further examine the causal influence of D2/D3 dopamine receptor antagonism on computational mechanisms governing intentional attributions within a simple game theoretic context. Using a formal model of intentional attributions and an iterative Dictator game (Barnby et al, 2020a; 2022a), we test the impact of haloperidol, a D2/D3 antagonist, and L-DOPA, a presynaptic dopamine potentiator, on paranoid beliefs using past data (Barnby et al, 2020b). Primarily we assessed whether haloperidol alters key computational processes involved in mental state inferences, allowing distinctions between trait representational changes (priors) and state learning processes (policy flexibility, uncertainty) along each attributional dimension (harmful intent and self-interest).…”

Aberrant perception of environmental volatility during social learning in emerging psychosis

“…Therefore, this mechanism may not be specifically tied to social contexts, but instead may be related to a more general deficit in learning under uncertainty (Reed et al, 2020; Suthaharan et al, 2021). However, we do note that the social control task employed by Suthaharan et al (2021) was not as ecologically valid as other tasks that were used to study paranoia such as the dictator game (Raihani and Bell, 2017; Barnby et al, 2020, 2022) or our task which was adapted from empirically-observed human-human interactions in a previous study using videos of human advisers intending to either help or deceive players (Diaconescu et al, 2014). Finally, it is also possible that there are both domain-general and domain-specific changes, but that these can only be studied at the neuronal level and converge on the same behavioural model parameters.…”

Assumed shared belief about conspiracy theories in social networks protects paranoid individuals against distress

“…But our data continue to suggest that paranoia and conspiracy theorizing-even the social aspects thereof-are related to low-level domain-general primary learning mechanisms 33 which may be dopaminergically and noradrenergically mediated 34,35 . This domain general contribution to paranoia involves learning expectancies about the environment and the agents within it 36 , a process that may be guided by using oneself as a source of prior beliefs 18 . Our data represent an extension of those processes into real world social relationships.…”