Deciding where to look based on visual, auditory, and semantic information

Abstract

Neurons in the dorsal frontal and parietal cortex are thought to transform incoming visual signals into the spatial goals of saccades, a process known as target selection. Here, we used functional magnetic resonance imaging (fMRI) to test how target selection may generalize beyond visual transformations when auditory and semantic information is used for selection. We compared activity in the frontal and parietal cortex when subjects made visually, aurally, and semantically guided saccades to one of fourts. Selection was based on a visual cue (i.e., one of the dots blinked), an auditory cue (i.e., a white noise burst was emitted at one of the dots’ location), or a semantic cue (i.e., the color of one of the dots was spoken). Although neural responses in frontal and parietal cortex were robust, they were non-specific with regard to the type of information used for target selection. Decoders, however, trained on the patterns of activity in the intraparietal sulcus could classify both the type of cue used for target selection and the direction of the saccade. Therefore, we find evidence that the posterior parietal cortex is involved in transforming multimodal inputs into general spatial representations that can be used to guide saccades.

Broad domain generality in focal regions of frontal and parietal cortex

“…Nevertheless, complementing our results, several studies have used multivoxel pattern analyses (86) to show that MD regions code many specific properties of attended stimuli, responses and tasks in the fine-grained patterns of spatial activity (e.g., refs. 32,[87][88][89][90][91][92][93]. Such results dovetail with analogous findings of widespread, adaptive coding of task-relevant information in single neurons of frontal and parietal cortex (8,12).…”

Differential roles of the frontal and parietal cortices in the control of saccades

“…This is not surprising as the subjects had time to prepare their saccades during the long delay periods, which often nullify latency differences (Curtis and Connolly, 2008). Moreover, significant differences in saccade latencies between the groups would have complicated interpreting the BOLD data during the response period (Tark & Curtis, in review). …”

Short-Term Audiovisual Spatial Training Enhances Electrophysiological Correlates of Auditory Selective Spatial Attention

“…This result may be compatible with previous findings indicating multi- or supramodal properties of the dorsofrontal networks that have been usually associated with selective spatial attention in the visual modality (e.g., Slotnick and Moo, 2006 ; Macaluso, 2010 ; Lee et al, 2012 ; Bharadwaj et al, 2014 ; Lewald et al, 2018 ). For the monkey DLPFC, it has been suggested that neuronal processes exist for visual and auditory location information and spatial working memory ( Fuster et al, 2000 ; Kikuchi-Yorioka and Sawaguchi, 2000 ; Artchakov et al, 2007 ; Hwang and Romanski, 2015 ; for review, see Plakke and Romanski, 2016 ), and the human DLPFC has been shown to be involved in transforming auditory and visual inputs into multimodal spatial representations that can be used to guide saccades ( Tark and Curtis, 2013 ). The monkey DLPFC receives projections from posterior auditory cortex areas known to be involved in spatial processing and from the posterior parietal cortex ( Chavis and Pandya, 1976 ; Rauschecker et al, 1995 ; Romanski et al, 1999a , b ).…”