Physical inference of falling objects involves simulation of occluded trajectories in early visual areas

Abstract

Humans possess an intuitive understanding of the environment's physical properties and dynamics, which allows them to predict the outcomes of physical scenarios and successfully interact with the physical world. This predictive ability is thought to rely on mental simulations and has been shown to involve frontoparietal areas. Here, we investigate whether such mental simulations may be accompanied by visual imagery of the predicted physical scene. We designed an intuitive physical inference task requiring participants to infer the parabolic trajectory of an occluded ball falling in accordance with Newtonian physics. Participants underwent fMRI while (i) performing the physical inference task alternately with a visually matched control task, and (ii) passively observing falling balls depicting the trajectories that had to be inferred during the physical inference task. We found that performing the physical inference task activates early visual areas together with a frontoparietal network when compared with the control task. Using multivariate pattern analysis, we show that these regions contain information specific to the trajectory of the occluded ball (i.e., fall direction), despite the absence of visual inputs. Using a cross‐classification approach, we further show that in early visual areas, trajectory‐specific activity patterns evoked by the physical inference task resemble those evoked by the passive observation of falling balls. Together, our findings suggest that participants simulated the ball trajectory when solving the task, and that the outcome of these simulations may be represented in form of the perceivable sensory consequences in early visual areas.

Inferring occluded projectile motion changes connectivity within a visuo-fronto-parietal network

“…The data used in this manuscript have previously been published in Zbären et al (2023), which primarily focused on univariate and multivariate pattern analysis of the BOLD signal. Here, we have re-analysed the same data with an emphasis on functional connectivity analysis and network modelling using DCM.…”

“…Here, we have re-analysed the same data with an emphasis on functional connectivity analysis and network modelling using DCM. The experimental paradigm and preprocessing pipeline are identical to Zbären et al (2023). We restate the relevant details here for the readers' convenience.…”

“…Twenty healthy volunteers participated in the study and four were excluded from the analyses (for detailed exclusion criteria, see Zbären et al 2023). The final sample consisted of sixteen participants (10 females; 6 males; mean age: 28.31 ± 9.26) with normal or corrected-to-normal vision.…”

“…that early visual areas are also involved in the process. These regions have been shown to contain representations of inferred physical scenarios that resemble those evoked by perception, suggesting that intuitive physical inference may be accompanied by visual simulation of the inferred scenario (Ahuja et al 2022;Zbären et al 2023). Interestingly, during explicitly instructed visual imagery, internally-generated visual cortex activity is modulated by frontoparietal areas similar to those involved in physical inference (Dentico et al 2014;Dijkstra et al 2017;Ishai et al 2000;Mechelli 2004), suggesting a potential role of these regions in generating perception-like images when inferring occluded physical scenes.…”

“…In a recent study, we investigated the neural representations associated with predicting the parabolic trajectory of an occluded ball falling under Newtonian physics (Zbären et al 2023). We designed an intuitive physical inference task requiring participants to infer the trajectory of an occluded ball falling parabolically.…”