Highlights d Biomimetic hybrid sensory encodings are perceived as highly natural d Biomimetic hybrid sensory encodings restore rich tactile sensitivity d Biomimetic hybrid sensory encodings improve manual dexterity and accuracy d Biomimetic hybrid sensory encodings enhance prosthesis embodiment

Tales of ghosts, wraiths, and other apparitions have been reported in virtually all cultures. The strange sensation that somebody is nearby when no one is actually present and cannot be seen (feeling of a presence, FoP) is a fascinating feat of the human mind, and this apparition is often covered in the literature of divinity, occultism, and fiction. Although it is described by neurological and psychiatric patients and healthy individuals in different situations, it is not yet understood how the phenomenon is triggered by the brain. Here, we performed lesion analysis in neurological FoP patients, supported by an analysis of associated neurological deficits. Our data show that the FoP is an illusory own-body perception with well-defined characteristics that is associated with sensorimotor loss and caused by lesions in three distinct brain regions: temporoparietal, insular, and especially frontoparietal cortex. Based on these data and recent experimental advances of multisensory own-body illusions, we designed a master-slave robotic system that generated specific sensorimotor conflicts and enabled us to induce the FoP and related illusory own-body perceptions experimentally in normal participants. These data show that the illusion of feeling another person nearby is caused by misperceiving the source and identity of sensorimotor (tactile, proprioceptive, and motor) signals of one's own body. Our findings reveal the neural mechanisms of the FoP, highlight the subtle balance of brain mechanisms that generate the experience of "self" and "other," and advance the understanding of the brain mechanisms responsible for hallucinations in schizophrenia.

No abstract

Neuroprosthetics research in amputee patients aims at developing new prostheses that move and feel like real limbs. Targeted muscle and sensory reinnervation (TMSR) is such an approach and consists of rerouting motor and sensory nerves from the residual limb towards intact muscles and skin regions. Movement of the myoelectric prosthesis is enabled via decoded electromyography activity from reinnervated muscles and touch sensation on the missing limb is enabled by stimulation of the reinnervated skin areas. Here we ask whether and how motor control and redirected somatosensory stimulation provided via TMSR affected the maps of the upper limb in primary motor (M1) and primary somatosensory (S1) cortex, as well as their functional connections. To this aim, we tested three TMSR patients and investigated the extent, strength, and topographical organization of the missing limb and several control body regions in M1 and S1 at ultra high-field (7 T) functional magnetic resonance imaging. Additionally, we analysed the functional connectivity between M1 and S1 and of both these regions with fronto-parietal regions, known to be important for multisensory upper limb processing. These data were compared with those of control amputee patients (n = 6) and healthy controls (n = 12). We found that M1 maps of the amputated limb in TMSR patients were similar in terms of extent, strength, and topography to healthy controls and different from non-TMSR patients. S1 maps of TMSR patients were also more similar to normal conditions in terms of topographical organization and extent, as compared to non-targeted muscle and sensory reinnervation patients, but weaker in activation strength compared to healthy controls. Functional connectivity in TMSR patients between upper limb maps in M1 and S1 was comparable with healthy controls, while being reduced in non-TMSR patients. However, connectivity was reduced between S1 and fronto-parietal regions, in both the TMSR and non-TMSR patients with respect to healthy controls. This was associated with the absence of a well-established multisensory effect (visual enhancement of touch) in TMSR patients. Collectively, these results show how M1 and S1 process signals related to movement and touch are enabled by targeted muscle and sensory reinnervation. Moreover, they suggest that TMSR may counteract maladaptive cortical plasticity typically found after limb loss, in M1, partially in S1, and in their mutual connectivity. The lack of multisensory interaction in the present data suggests that further engineering advances are necessary (e.g. the integration of somatosensory feedback into current prostheses) to enable prostheses that move and feel as real limbs.

Experimental manipulations of body ownership have indicated that multisensory integration is central to forming bodily self-representation. Voluntary self-touch is a unique multisensory situation involving corresponding motor, tactile and proprioceptive signals. Yet, even though self-touch is frequent in everyday life, its contribution to the formation of body ownership is not well understood. Here we investigated the role of voluntary self-touch in body ownership using a novel adaptation of the rubber hand illusion (RHI), in which a robotic system and virtual reality allowed participants self-touch of real and virtual hands. In the first experiment, active and passive self-touch were applied in the absence of visual feedback. In the second experiment, we tested the role of visual feedback in this bodily illusion. Finally, in the third experiment, we compared active and passive self-touch to the classical RHI in which the touch is administered by the experimenter. We hypothesized that active self-touch would increase ownership over the virtual hand through the addition of motor signals strengthening the bodily illusion. The results indicated that active self-touch elicited stronger illusory ownership compared to passive self-touch and sensory only stimulation, and show an important role for active self-touch in the formation of bodily self.

Hallucinations in Parkinson’s disease (PD) are disturbing and frequent non-motor symptoms and constitute a major risk factor for psychosis and dementia. We report a robotics-based approach applying conflicting sensorimotor stimulation, enabling the induction of presence hallucinations (PHs) and the characterization of a subgroup of patients with PD with enhanced sensitivity for conflicting sensorimotor stimulation and robot-induced PH. We next identify the fronto-temporal network of PH by combining MR-compatible robotics (and sensorimotor stimulation in healthy participants) and lesion network mapping (neurological patients without PD). This PH-network was selectively disrupted in an additional and independent cohort of patients with PD, predicted the presence of symptomatic PH, and associated with cognitive decline. These robotics-neuroimaging findings extend existing sensorimotor hallucination models to PD and reveal the pathological cortical sensorimotor processes of PH in PD, potentially indicating a more severe form of PD that has been associated with psychosis and cognitive decline.

The effects of real-world tool use on body or space representations are relatively well established in cognitive neuroscience. Several studies have shown, for example, that active tool use results in a facilitated integration of multisensory information in peripersonal space, i.e. the space directly surrounding the body. However, it remains unknown to what extent similar mechanisms apply to the use of virtual-robotic tools, such as those used in the field of surgical robotics, in which a surgeon may use bimanual haptic interfaces to control a surgery robot at a remote location. This paper presents two experiments in which participants used a haptic handle, originally designed for a commercial surgery robot, to control a virtual tool. The integration of multisensory information related to the virtual-robotic tool was assessed by means of the crossmodal congruency task, in which subjects responded to tactile vibrations applied to their fingers while ignoring visual distractors superimposed on the tip of the virtual-robotic tool. Our results show that active virtual-robotic tool use changes the spatial modulation of the crossmodal congruency effects, comparable to changes in the representation of peripersonal space observed during real-world tool use. Moreover, when the virtual-robotic tools were held in a crossed position, the visual distractors interfered strongly with tactile stimuli that was connected with the hand via the tool, reflecting a remapping of peripersonal space. Such remapping was not only observed when the virtual-robotic tools were actively used (Experiment 1), but also when passively held the tools (Experiment 2). The present study extends earlier findings on the extension of peripersonal space from physical and pointing tools to virtual-robotic tools using techniques from haptics and virtual reality. We discuss our data with respect to learning and human factors in the field of surgical robotics and discuss the use of new technologies in the field of cognitive neuroscience.

Abstract Dysfunction of sensorimotor predictive processing is thought to underlie abnormalities in self-monitoring producing passivity symptoms in psychosis. Experimentally induced sensorimotor conflict can produce a failure in bodily self-monitoring (presence hallucination [PH]), yet it is unclear how this is related to auditory self-monitoring and psychosis symptoms. Here we show that the induction of sensorimotor conflict in early psychosis patients induces PH and impacts auditory-verbal self-monitoring. Participants manipulated a haptic robotic system inducing a bodily sensorimotor conflict. In experiment 1, the PH was measured. In experiment 2, an auditory-verbal self-monitoring task was performed during the conflict. Fifty-one participants (31 early psychosis patients, 20 matched controls) participated in the experiments. The PH was present in all participants. Psychosis patients with passivity experiences (PE+) had reduced accuracy in auditory-verbal self-other discrimination during sensorimotor stimulation, but only when sensorimotor stimulation involved a spatiotemporal conflict (F(2, 44) = 6.68, P = .002). These results show a strong link between robotically controlled alterations in sensorimotor processing and auditory misattribution in psychosis and provide evidence for the role of sensorimotor processes in altered self-monitoring in psychosis.