A Brain-Machine Interface Enables Bimanual Arm Movements in Monkeys

Abstract

Brain-machine interfaces (BMIs) are artificial systems that aim to restore sensation and movement to severely paralyzed patients. However, previous BMIs enabled only single arm functionality, and control of bimanual movements was a major challenge. Here, we developed and tested a bimanual BMI that enabled rhesus monkeys to control two avatar arms simultaneously. The bimanual BMI was based on the extracellular activity of 374–497 neurons recorded from several frontal and parietal cortical areas of both cerebral hemispheres. Cortical activity was transformed into movements of the two arms with a decoding algorithm called a 5th order unscented Kalman filter (UKF). The UKF is well-suited for BMI decoding because it accounts for both characteristics of reaching movements and their representation by cortical neurons. The UKF was trained either during a manual task performed with two joysticks or by having the monkeys passively observe the movements of avatar arms. Most cortical neurons changed their modulation patterns when both arms were engaged simultaneously. Representing the two arms jointly in a single UKF decoder resulted in improved decoding performance compared with using separate decoders for each arm. As the animals’ performance in bimanual BMI control improved over time, we observed widespread plasticity in frontal and parietal cortical areas. Neuronal representation of the avatar and reach targets was enhanced with learning, whereas pairwise correlations between neurons initially increased and then decreased. These results suggest that cortical networks may assimilate the two avatar arms through BMI control.

Single-unit activity, threshold crossings, and local field potentials in motor cortex differentially encode reach kinematics

“…Our ability to simultaneously record the activity of single neurons has increased steadily with time (Stevenson and Kording 2011) and multielectrode recording arrays currently allows us to record from several hundred neurons at the same time (Collinger et al 2013;Ifft et al 2013;Schwarz et al 2014). However, this still only represents a small fraction of the total number of neurons in any given brain area.…”

“…Our results have implications for BMIs, devices that use recorded neural signals to actuate some device, such as a computer cursor (Taylor et al 2002;Suminski et al 2009;Ganguly and Carmena 2009;Hochberg et al 2006;Mulliken et al 2008;Schalk et al 2007;Gilja et al 2012;Ifft et al 2013), robotic arm (Collinger et al 2013;Hochberg et al 2012;Wang et al 2013), or muscle stimulator (Ethier et al 2012;Moritz et al 2008). TC activity is often thought to convey similar information as SUA.…”

Large-scale recording of thalamocortical circuits: in vivo electrophysiology with the two-dimensional electronic depth control silicon probe

“…Over the past few decades, single-wire electrodes used for in vivo extracellular recording of action potentials evolved into multielectrode arrays covering the range of up to 1,000 recording sites (Bai and Wise 2001;Berényi et al 2014;Blanche et al 2005;Bragin et al 2000;Campbell et al 1991;Chen et al 2009;Cheung 2007;Csicsvari et al 2003;Drake et al 1988;Du et al 2009Du et al , 2011Grand et al 2011;Karmos et al 1982;Khodagholy et al 2015;Kipke et al 2008;Kubie 1984;Lopez et al 2014;Márton et al 2015;McNaughton et al 1983;Michon et al 2016;Okeefe and Recce 1993;Ruther et al 2010;Ruther and Paul 2015;Scholvin et al 2016;Seidl et al 2011Seidl et al , 2012Shobe et al 2015;Torfs et al 2011;Wilson and McNaughton 1993;Wise et al 1970Wise et al , 2008. With such a high number of recording sites neuroscientists are able to monitor the activity of hundreds of neurons simultaneously both in anesthetized and in freely moving animals (Berényi et al 2014;Ifft et al 2013;Nicolelis et al 2003;Vandecasteele et al 2012), which is fundamental for the understanding of complex neuronal computations and higher order cognitive functions, such as learning, memory, or language (Buzsáki 2004). In addition, the reliability and performance of current invasive brain-machine interfaces, aim...…”

Brain-machine interfaces: an overview

“…Neurons engaged in BMI control exhibit stronger modulations [63], changes in correlation with each other [6,29], and changes in directional tuning [27]. Additionally, cortical neurons have been shown to adapt to rotational transformation applied to a subset of neurons engaged in BMI control [64].…”

“…In the meanwhile the group of Nicolelis developed a BMI that controlled two virtual arms simultaneously [29]. Several hundreds of neurons of neurons were sampled simultaneously in multiple cortical areas.…”