Spatiotemporal transformations for gaze control

Abstract

Sensorimotor transformations require spatiotemporal coordination of signals, that is, through both time and space. For example, the gaze control system employs signals that are time‐locked to various sensorimotor events, but the spatial content of these signals is difficult to assess during ordinary gaze shifts. In this review, we describe the various models and methods that have been devised to test this question, and their limitations. We then describe a new method that can (a) simultaneously test between all of these models during natural, head‐unrestrained conditions, and (b) track the evolving spatial continuum from target (T) to future gaze coding (G, including errors) through time. We then summarize some applications of this technique, comparing spatiotemporal coding in the primate frontal eye field (FEF) and superior colliculus (SC). The results confirm that these areas preferentially encode eye‐centered, effector‐independent parameters, and show—for the first time in ordinary gaze shifts—a spatial transformation between visual and motor responses from T to G coding. We introduce a new set of spatial models (T‐G continuum) that revealed task‐dependent timing of this transformation: progressive during a memory delay between vision and action, and almost immediate without such a delay. We synthesize the results from our studies and supplement it with previous knowledge of anatomy and physiology to propose a conceptual model where cumulative transformation noise is realized as inaccuracies in gaze behavior. We conclude that the spatiotemporal transformation for gaze is both local (observed within and across neurons in a given area) and distributed (with common signals shared across remote but interconnected structures).

Spatiotemporal Coding in the Macaque Supplementary Eye Fields: Landmark Influence in the Target-to-Gaze Transformation

“…Previous reports on FEF responses from our lab have reported that responses do not fit exactly against spatial models like Te or Ge, but actually may fit best against intermediate models between the canonical ones (Fig. 4A, lower left) (Sajad et al, 2020). As in our previous studies (Sadeh et al, 2020;Sajad et al, 2016), we found that a T-G continuum (specifically, steps along the 'error line' between Te to Ge) best quantified the SEF egocentric transformation (Fig.…”

“…Note that in the current study, deviations of gaze from the target (used to fit response fields against G) were produced in part by the landmark shift. However, this alone does not likely explain the T-G transition in our cells, because it happened continuously through the task, it was spatially separable and often uncorrelated with the neural response to the landmark shift (discussed below), and much of the gaze errors used to calculate this transition were not due to the landmark shift, but appeared to be due to general internal 'noise', as in our previous studies (Sadeh et al, 2020;Sajad et al, 2020Sajad et al, , 2016 which was even larger without a landmark (Li et al, 2017). In short, the landmark shift clearly contributed to gaze errors, but cannot alone explain the T-G transition we observed here in SEF cells.…”

“…As we shall see (Fig. 1 C), AW scores for individual trials often fell between 0 and 1 but varied considerably, possibly due to trial-to-trial variations in landmark influence and / or other sources of variable gaze error that are present without a landmark shift (Sajad et al, 2020(Sajad et al, , 2016(Sajad et al, , 2015.…”

Landmark-Centered Coding in Frontal Cortex Visual Responses

“…This occurs when the ideal coordinate frame is not used for the plot (Fig. 3b), but even for the best model, some variability (residuals) persists due to non-spatial factors such as attention and motivation 6,7,33,34 . For the first step in our new analysis, we separated and plotted the response fields for each of the four target-landmark configurations (Fig.…”

Saccades along spatial neural circuit discontinuities

“…Saccade generation involves multiple transformations of reference frames in neurons (Crawford and Guitton 1997;Goossens and van Opstal 2012;Leigh et al 1997;Sajad et al 2020). For example, visual targets for saccades are represented by retinocentric sensory maps, and gaze shift commands are initially also specified as desired displacement vectors in retinocentric coordinates (Goldberg and Bruce 1990;Hepp et al 1993; Moschovakis and Highstein 1994;Robinson 1972;Schiller and Stryker 1972;Schlag-Rey et al 1989;van Opstal et al 1991).…”