Restoring the Sense of Touch Using a Sensorimotor Demultiplexing Neural Interface

Ganzer, Patrick D.; th, Samuel C. Colachis; Schwemmer, Michael A.; Friedenberg, David A.; Dunlap, Collin F.; Swiftney, Carly E.; Jacobowitz, Adam F.; Weber, Doug; Bockbrader, Marcia; Sharma, Gaurav

doi:10.1016/j.cell.2020.03.054

Cited by 107 publications

(81 citation statements)

References 88 publications

(210 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accurate but slow movements can be generated for high-dimensional artificial hands with wrist and digits; however, the accuracy is challenged by changing limb posture and orientation. One potential solution is the use of closed-loop control systems that provide not only the forward control of prosthetic, but also incorporate the sensory feedback within neuroprosthetics (Charkhkar et al, 2020; Ganzer et al, 2020; Hughes et al, 2020). The closed-loop control system that takes into account muscle forces will likely require accurate representation of musculoskeletal actions, which are essential for the description of both muscle forces and sensory signals responsible for proprioception.…”

Section: Discussionmentioning

confidence: 99%

Solving musculoskeletal biomechanics with machine learning

Smirnov

Попов

et al. 2020

Preprint

View full text Add to dashboard Cite

Deep learning is a relatively new computational technique for the description of the musculoskeletal dynamics. The experimental relationships of muscle geometry in different postures are the high-dimensional spatial transformations that can be approximated by relatively simple functions, which opens the opportunity for machine learning applications. In this study, we challenged general machine learning algorithms with the problem of approximating the posture-dependent moment arm and muscle length relationships of the human arm and hand muscles. We used two types of algorithms, light gradient boosting machine (LGB) and fully connected artificial neural network (ANN) solving the wrapping kinematics of 33 muscles spanning up to six degrees of freedom (DOF) each for the arm and hand model with 18 DOFs. The input-output training and testing datasets were generated by our previous phenomenological model based on the autogenerated polynomial structures (Sobinov et al., 2019). Both models achieved a similar level of errors: ANN model errors were 0.08±0.05% for muscle lengths and 0.53±0.29% for moment arms, and LGB model made similar errors--0.18±0.06% and 0.13±0.07%, respectively. LGB model reached the training goal with only 10^3 samples, while ANN required 10^6 samples; however, LGB models were about 39 slower than ANN models in the evaluation. The sufficient performance of developed models demonstrates the future applicability of machine learning for musculoskeletal transformations in a variety of applications, such as in advanced powered prosthetics.

show abstract

Section: Discussionmentioning

confidence: 99%

Solving musculoskeletal biomechanics with machine learning

Smirnov

Попов

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…However, recent studies have demonstrated retained sensory pathways in some patients deemed clinically complete SCI, termed sensory discomplete. [31][32][33] Patients with clinically complete injuries could be evaluated prior to surgery for the presence of sensory discomplete lesions (e.g, fMRI activity present for finger sensations), which could benefit from the same approach described in this study.…”

Section: Limitations and Next Stepsmentioning

confidence: 99%

Novel intraoperative online functional mapping of somatosensory finger representations for targeted stimulating electrode placement

McMullen

Thomas

Fifer

et al. 2020

Preprint

View full text Add to dashboard Cite

Defining eloquent cortex intraoperatively, traditionally performed by neurosurgeons to preserve patient function, can now help target electrode implantation for restoring function. Brain-machine interfaces (BMIs) have the potential to restore upper-limb motor control to paralyzed patients but require accurate placement of recording and stimulating electrodes to enable functional control of a prosthetic limb. Beyond motor decoding from recording arrays, precise placement of stimulating electrodes in cortical areas associated with finger and fingertip sensations allows for the delivery of sensory feedback that could improve dexterous control of prosthetic hands. In our study, we demonstrated the use of a novel intraoperative online functional mapping (OFM) technique with high-density electrocorticography (ECoG) to localize finger representations in human primary somatosensory cortex. In conjunction with traditional pre- and intraoperative targeting approaches, this technique enabled accurate implantation of stimulating microelectrodes, which was confirmed by post-implantation intra-cortical stimulation of finger and fingertip sensations. This work demonstrates the utility of intraoperative OFM and will inform future studies of closed-loop BMIs in humans.

show abstract

“…Such fine-resolution recordings have enabled intuitive control of high DOF robotic arms (Wodlinger et al, 2015 ) and functional electrical stimulation (FES) systems (Colachis et al, 2018 ) by people with quadriplegia. MEAs can also provide the necessary somatosensory feedback-control to enhance grip performance of high DOF robotic arms (Flesher et al, 2017 ) and advanced FES orthotics (Ganzer et al, 2020 ). This review focuses on recording disruptions that affect MEA BMI performance and limit their use.…”

Section: Introductionmentioning

confidence: 99%

Classifying Intracortical Brain-Machine Interface Signal Disruptions Based on System Performance and Applicable Compensatory Strategies: A Review

et al. 2020

Self Cite

View full text Add to dashboard Cite

Brain-machine interfaces (BMIs) record and translate neural activity into a control signal for assistive or other devices. Intracortical microelectrode arrays (MEAs) enable high degree-of-freedom BMI control for complex tasks by providing fine-resolution neural recording. However, chronically implanted MEAs are subject to a dynamic in vivo environment where transient or systematic disruptions can interfere with neural recording and degrade BMI performance. Typically, neural implant failure modes have been categorized as biological, material, or mechanical. While this categorization provides insight into a disruption's causal etiology, it is less helpful for understanding degree of impact on BMI function or possible strategies for compensation. Therefore, we propose a complementary classification framework for intracortical recording disruptions that is based on duration of impact on BMI performance and requirement for and responsiveness to interventions: (1) Transient disruptions interfere with recordings on the time scale of minutes to hours and can resolve spontaneously; (2) Reversible disruptions cause persistent interference in recordings but the root cause can be remedied by an appropriate intervention; (3) Irreversible compensable disruptions cause persistent or progressive decline in signal quality, but their effects on BMI performance can be mitigated algorithmically; and (4) Irreversible non-compensable disruptions cause permanent signal loss that is not amenable to remediation or compensation. This conceptualization of intracortical BMI disruption types is useful for highlighting specific areas for potential hardware improvements and also identifying opportunities for algorithmic interventions. We review recording disruptions that have been reported for MEAs and demonstrate how biological, material, and mechanical mechanisms of disruption can be further categorized according to their impact on signal characteristics. Then we discuss potential compensatory protocols for each of the proposed disruption classes. Specifically, transient disruptions may be minimized by using robust neural decoder features, data augmentation methods, adaptive machine learning models, and specialized signal referencing techniques. Statistical Process Control methods can identify reparable disruptions for rapid intervention. In-vivo diagnostics such as impedance spectroscopy can inform neural feature selection and decoding models to compensate for irreversible disruptions. Additional compensatory strategies for irreversible disruptions include information salvage techniques, data augmentation during decoder training, and adaptive decoding methods to down-weight damaged channels.

show abstract

Restoring the Sense of Touch Using a Sensorimotor Demultiplexing Neural Interface

Cited by 107 publications

References 88 publications

Solving musculoskeletal biomechanics with machine learning

Solving musculoskeletal biomechanics with machine learning

Novel intraoperative online functional mapping of somatosensory finger representations for targeted stimulating electrode placement

Classifying Intracortical Brain-Machine Interface Signal Disruptions Based on System Performance and Applicable Compensatory Strategies: A Review

Contact Info

Product

Resources

About