We demonstrate five-degree-of-freedom (5-DOF) wireless magnetic control of a fully untethered microrobot (3-DOF position and 2-DOF pointing orientation). The microrobot can move through a large workspace and is completely unrestrained in the rotation DOF. We accomplish this level of wireless control with an electromagnetic system that we call OctoMag. OctoMag's unique abilities are due to its utilization of complex nonuniform magnetic fields, which capitalizes on a linear representation of the coupled field contributions of multiple soft-magnetic-core electromagnets acting in concert. OctoMag was primarily designed to control intraocular microrobots for delicate retinal procedures, but it also has potential uses in other medical applications or micromanipulation under an optical microscope.

The effects of real-world tool use on body or space representations are relatively well established in cognitive neuroscience. Several studies have shown, for example, that active tool use results in a facilitated integration of multisensory information in peripersonal space, i.e. the space directly surrounding the body. However, it remains unknown to what extent similar mechanisms apply to the use of virtual-robotic tools, such as those used in the field of surgical robotics, in which a surgeon may use bimanual haptic interfaces to control a surgery robot at a remote location. This paper presents two experiments in which participants used a haptic handle, originally designed for a commercial surgery robot, to control a virtual tool. The integration of multisensory information related to the virtual-robotic tool was assessed by means of the crossmodal congruency task, in which subjects responded to tactile vibrations applied to their fingers while ignoring visual distractors superimposed on the tip of the virtual-robotic tool. Our results show that active virtual-robotic tool use changes the spatial modulation of the crossmodal congruency effects, comparable to changes in the representation of peripersonal space observed during real-world tool use. Moreover, when the virtual-robotic tools were held in a crossed position, the visual distractors interfered strongly with tactile stimuli that was connected with the hand via the tool, reflecting a remapping of peripersonal space. Such remapping was not only observed when the virtual-robotic tools were actively used (Experiment 1), but also when passively held the tools (Experiment 2). The present study extends earlier findings on the extension of peripersonal space from physical and pointing tools to virtual-robotic tools using techniques from haptics and virtual reality. We discuss our data with respect to learning and human factors in the field of surgical robotics and discuss the use of new technologies in the field of cognitive neuroscience.

Although there is increasing knowledge about how visual and tactile cues from the hands are integrated, little is known about how self-generated hand movements affect such multisensory integration. Visuo-tactile integration often occurs under highly dynamic conditions requiring sensorimotor updating. Here, we quantified visuo-tactile integration by measuring cross-modal congruency effects (CCEs) in different bimanual hand movement conditions with the use of a robotic platform. We found that classical CCEs also occurred during bimanual self-generated hand movements, and that such movements lowered the magnitude of visuo-tactile CCEs as compared to static conditions. Visuo-tactile integration, body ownership and the sense of agency were decreased by adding a temporal visuo-motor delay between hand movements and visual feedback. These data show that visual stimuli interfere less with the perception of tactile stimuli during movement than during static conditions, especially when decoupled from predictive motor information. The results suggest that current models of visuo-tactile integration need to be extended to account for multisensory integration in dynamic conditions.

This paper presents a novel lightweight and simple TSA-based (twisted string actuation) wearable haptic glove (ExoTen-Glove). This system is using two independent twisted string actuators with integrated force sensors and small-size DC motors. The proposed system can provide users force feedback during the execution of grasping virtual objects. The design of the TSA-based ExoTen-Glove, description of the TSA system, the controller and the preliminary experimental evaluation of the proposed system has been presented in this paper. This device has been evaluated by an experiment in virtual reality environment using HTC VIVE headset with 2 degrees of freedom grasping tasks, where the participants were squeezing a real spring with their thumb and index finger and compare it with a virtual spring stiffness. The results prove the applicability of the ExoTen-Glove for rehabilitation and haptic purposes.

In the first experiment it was found that tool-use training with force feedback facilitates multisensory integration of signals from the tool, as reflected in a stronger crossmodal congruency effect with the force feedback training compared to training without force feedback and to no training. The second experiment extends these findings by showing that training with realistic online force feedback resulted in a stronger crossmodal congruency effect compared to training in which force feedback was delayed. The present study highlights the importance of haptic information for multisensory integration and extends findings from classical tool-use studies to the domain of robotic tools. We argue that such crossmodal congruency effects are an objective measure of robotic tool integration and propose some potential applications in surgical robotics, robotic tools, and human-tool interaction.