A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping

Abstract

One of the primary ways we interact with the world is using our hands. In macaques, the circuit spanning the anterior intraparietal area, the hand area of the ventral premotor cortex, and the primary motor cortex is necessary for transforming visual information into grasping movements. However, no comprehensive model exists that links all steps of processing from vision to action. We hypothesized that a recurrent neural network mimicking the modular structure of the anatomical circuit and trained to use visual features of objects to generate the required muscle dynamics used by primates to grasp objects would give insight into the computations of the grasping circuit. Internal activity of modular networks trained with these constraints strongly resembled neural activity recorded from the grasping circuit during grasping and paralleled the similarities between brain regions. Network activity during the different phases of the task could be explained by linear dynamics for maintaining a distributed movement plan across the network in the absence of visual stimulus and then generating the required muscle kinematics based on these initial conditions in a module-specific way. These modular models also outperformed alternative models at explaining neural data, despite the absence of neural data during training, suggesting that the inputs, outputs, and architectural constraints imposed were sufficient for recapitulating processing in the grasping circuit. Finally, targeted lesioning of modules produced deficits similar to those observed in lesion studies of the grasping circuit, providing a potential model for how brain regions may coordinate during the visually guided grasping of objects.

From Observed Action Identity to Social Affordances

“…repertoire. Importantly, recent simultaneous recordings from multiple areas in animal models and the application of dynamical system frameworks to the analysis of neuronal populations data [82,83] greatly contributed to elucidate the visuomotor transformations underlying the identification and selection of object features relevant for action planning and execution [20,84]; these approaches will likely play an important role in deciphering the neural and computational principles underlying social affordance processing in different contexts (see Outstanding Questions).…”

Modeling of hyper-adaptability: from motor coordination to rehabilitation

“…This approach can be divided into two steps: (1) building network models to reproduce intact behavior (forward engineering) and (2) 'breaking' the model to analyze the internal structure and emulate the motor impairments and recovery processes (reverse engineering). Michaels et al (2020) showed that the inhibition of a part of hierarchical RNNs showed unique patterns of motor deficits similar to those observed in animal experiments [69]. Other studies also demonstrated that deactivation of brain areas could be emulated by the deactivation of motor control models, such as the optimal feedback control model [70,71].…”

Next-generation of recurrent neural network models for cognition

“…Utilizing multiple areas in RNNs is technically straightforward and commonplace in machine learning [28,29]. In neuroscience, multi-area RNNs models are increasingly being used to address specific questions about how different circuits in the brain interact [30][31][32][33][34]. The main challenge is to have different model areas meaningfully map onto actual brain areas.…”

“…Another approach is based on area-specific connectivity. For example, one can build a hierarchy of RNNs, where the first area preferentially targets the second, and so on, similar to hierarchical models of the sensory systems (Fig 1c) [30,32]. In a hierarchical RNN, modules lower on the hierarchy will have representation/dynamics resembling more the (sensory) inputs, while those higher along the hierarchy will have representations more closely related to the (motor) outputs [30].…”