A fundamental question in sensorimotor control concerns the transformation of spatial signals from the retina into eye and head motor commands required for accurate gaze shifts. Here, we investigated these transformations by identifying the spatial codes embedded in visually evoked and movement-related responses in the frontal eye fields (FEFs) during head-unrestrained gaze shifts. Monkeys made delayed gaze shifts to the remembered location of briefly presented visual stimuli, with delay serving to dissociate visual and movement responses. A statistical analysis of nonparametric model fits to response field data from 57 neurons (38 with visual and 49 with movement activities) eliminated most effector-specific, head-fixed, and space-fixed models, but confirmed the dominance of eye-centered codes observed in head-restrained studies. More importantly, the visual response encoded target location, whereas the movement response mainly encoded the final position of the imminent gaze shift (including gaze errors). This spatiotemporal distinction between target and gaze coding was present not only at the population level, but even at the single-cell level. We propose that an imperfect visual–motor transformation occurs during the brief memory interval between perception and action, and further transformations from the FEF's eye-centered gaze motor code to effector-specific codes in motor frames occur downstream in the subcortical areas.

To explore the possible cortical mechanisms underlying the 3-dimensional (3D) visuomotor transformation for reaching, we trained a 4-layer feed-forward artificial neural network to compute a reach vector (output) from the visual positions of both the hand and target viewed from different eye and head orientations (inputs). The emergent properties of the intermediate layers reflected several known neurophysiological findings, for example, gain field-like modulations and position-dependent shifting of receptive fields (RFs). We performed a reference frame analysis for each individual network unit, simulating standard electrophysiological experiments, that is, RF mapping (unit input), motor field mapping, and microstimulation effects (unit outputs). At the level of individual units (in both intermediate layers), the 3 different electrophysiological approaches identified different reference frames, demonstrating that these techniques reveal different neuronal properties and suggesting that a comparison across these techniques is required to understand the neural code of physiological networks. This analysis showed fixed input-output relationships within each layer and, more importantly, within each unit. These local reference frame transformation modules provide the basic elements for the global transformation; their parallel contributions are combined in a gain field-like fashion at the population level to implement both the linear and nonlinear elements of the 3D visuomotor transformation.

How we perceive the visual world as stable and unified suggests the existence of transsaccadic integration that retains and integrates visual information from one eye fixation to another eye fixation across saccadic eye movements. However, the capacity of transsaccadic integration is still a subject of controversy. We tested our subjects' memory capacity of two basic visual features, i.e. luminance (Experiment 1) and orientation (Experiment 2), both within a single fixation (i.e. visual working memory) and between separate fixations (i.e. transsaccadic memory). Experiment 2 was repeated, but attention allocation was manipulated using attentional cues at either the target or distracter (Experiment 3). Subjects were able to retain 3-4 objects in transsaccadic memory for luminance and orientation; errors generally increased as saccade size increased; and, subjects were more accurate when attention was allocated to the same location as the impending target. These results were modelled by inputting a noisy extra-retinal signal into an eye-centered feature map. Our results suggest that transsaccadic memory has a similar capacity for storing simple visual features as basic visual memory, but this capacity is dependent both on the metrics of the saccade and allocation of attention.