Paradigm Shift in Sensorimotor Control Research and Brain Machine Interface Control: The Influence of Context on Sensorimotor Representations

Abstract

Neural activity in the primary motor cortex (M1) is known to correlate with movement related variables including kinematics and dynamics. Our recent work, which we believe is part of a paradigm shift in sensorimotor research, has shown that in addition to these movement related variables, activity in M1 and the primary somatosensory cortex (S1) are also modulated by context, such as value, during both active movement and movement observation. Here we expand on the investigation of reward modulation in M1, showing that reward level changes the neural tuning function of M1 units to both kinematic as well as dynamic related variables. In addition, we show that this reward-modulated activity is present during brain machine interface (BMI) control. We suggest that by taking into account these context dependencies of M1 modulation, we can produce more robust BMIs. Toward this goal, we demonstrate that we can classify reward expectation from M1 on a movement-by-movement basis under BMI control and use this to gate multiple linear BMI decoders toward improved offline performance. These findings demonstrate that it is possible and meaningful to design a more accurate BMI decoder that takes reward and context into consideration. Our next step in this development will be to incorporate this gating system, or a continuous variant of it, into online BMI performance.

Motor Cortex Encodes A Temporal Difference Reinforcement Learning Process

“…This work was subsequently supported by Ramakrishnan et al 2017 (15) where they also showed that M1 and S1 responded to reward prediction errors and that reward could modulate sensorimotor directional tuning curves. We recently showed that this reward modulation of directional tuning holds during intracortical brain computer interface control (iBCI) when utilizing M1 (23), and that M1 grip force tuning functions are modulated by reward during manual control (23). In addition, we have recently discovered that reward modulation influences the relationship in M1 between the local field and individual neurons, where the spike field coherence is upregulated during nonrewarding trials as is the phase amplitude coupling between alpha phase and gamma amplitude (24).…”

“…Additionally, neural input to BMIs change with learning and time due to inherent instabilities such as the loss of single units or addition of new units, as well as due to the learning of state values as described in this report. It is also known that such value encoding changes M1 representations of directional tuning and grip-force turning functions (23). The concept of reinforcement learning may prove to be useful in transitioning BMIs to novel and unstable environments (32)(33)(34)(35)(36), as RL doesn't require a full error signal such as that used for supervised learning to update BMIs.…”

Reward Modulates Local Field Potentials, Spiking Activity and Spike-Field Coherence in the Primary Motor Cortex

“…These findings imply that alpha oscillations modulated by reward expectation have an influence on spike firing rate and spike timing during both reaching and grasping tasks in M1. These LFP, spike, and spike-field interactions could be used to follow the M1 neural state in order to enhance BMI decoding Zhao et al, 2018).…”

“…Furthermore, we found that an increase in alpha-band power was correlated with a decrease in neural spiking activity, that firing rates were highest at the trough of the alpha-band cycle and lowest at the peak of its cycle.These findings imply that alpha oscillations modulated by reward expectation have an influence on spike firing rate and spike timing during both reaching and grasping tasks in M1. These LFP, spike, and spike-field interactions could be used to follow the M1 neural state in order to enhance BMI decoding Zhao et al, 2018).…”

“…These LFP, spike, and spike-field interactions could be used to follow the M1 neural state in order to enhance BMI decoding Zhao et al, 2018).…”