Category-selective representation of relationships in visual cortex

Abstract

Understanding social interaction requires processing social agents and their relationship. Latest results show that much of this process is visually solved: visual areas can represent multiple people encoding emergent information about their interaction that is not explained by the response to the individuals alone. A neural signature of this process is an increased response in visual areas, to face-to-face (seemingly interacting) people, relative to people presented as unrelated (back-to-back). This effect highlighted a network of visual areas for representing relational information. How is this network organized? Using functional MRI, we measured brain activity of healthy female and male humans (N=42), in response to images of two faces or two (head-blurred) bodies, facing toward or away from each other. Taking the facing>non-facing effect as signature of relation perception, we found that relations between faces and between bodies were coded in distinct areas, mirroring the categorical representation of faces and bodies in visual cortex. Additional analyses suggest the existence of a third network encoding relations between (non-social) objects. Finally, a separate occipitotemporal network showed generalization of relational information across body, face and non-social object dyads (multivariate-pattern classification analysis), revealing shared properties of relations across categories. In sum, beyond single entities, visual cortex encodes the relations that bind multiple entities into relationships; it does so in a category-selective fashion, thus respecting a general organizing principle of representation in high-level vision. Visual areas encoding visual relational information can reveal the processing of emergent properties of social (and non-social) interaction which trigger inferential processes.

Converging evidence that left extrastriate body area supports visual sensitivity to social interactions

“…We first performed a new whole-brain fMRI analysis by combining data from 92 participants in three independent fMRI studies (Abassi & Papeo, 2020, 2022, 2023). In each of these studies, participants saw the same critical conditions (facing and non-facing body dyads, Figure 1A ).…”

“…Here, we have focused on the role of EBA in interaction perception. The fusiform body area (FBA) also shows some sensitivity to cues to dyadic interactions (Supplemental materials; Abassi & Papeo, 2020, 2023) yet the causal role for this ventral region on the 2BIE could not be directly assessed as it is not reachable by TMS. Additionally, previous evidence suggests that the fMRI sensitivity for facing dyads measured in the FBA, unlike the EBA, is not reliably related to the 2BIE (Abassi & Papeo, 2022).…”

“…We combined data from 92 unique participants (52 females and 40 males, self-reported sex; mean age 24.8 years, SD = 3.7) included in one of three previous independent fMRI studies ( N 20 from Abassi and Papeo, 2020; N 30 from Abassi & Papeo, 2022; N 42 from Abassi & Papeo, 2023). All participants had normal or corrected-to-normal vision, reported no history of psychiatric or neurological disorders, or use of psychoactive medications.…”

“…We used Daz3D (Daz Productions, Salt Lake City) and the Image Processing Toolbox of MATLAB (The MathWorks Inc, Natick, Massachusetts) to create and edit stimuli. For a full description of the stimuli, we refer the reader to Abassi & Papeo (2020; 2022; 2023).…”

“…This task was used to minimize eye movements and maintain vigilance in the scanner. For more information on the specificities of each study, we refer the reader to Abassi & Papeo (2020; 2022; 2023).…”