This paper presents a novel multimodal virtual rehabilitation environment. Its design and implementation are based on principles related to intrinsic motivation and game design. The system consists of visual, acoustic, and haptic modalities. Elements contributing to intrinsic motivation are carefully joined in the three modalities to increase patients' motivation during the long process of rehabilitation. The message in a bottle (MIB) virtual scenario is designed to allow interplay between motor and cognitive challenges in the exercising patient. The user first needs to perform a motor action to receive a cognitive challenge that is finally solved by a second motor action. Visual feedback provides the most relevant information related to the task. Acoustic feedback consists of environmental sounds, music, and spoken instructions or encouraging statements for the patient. The haptic modality generates tactile information related to the environment and provides various modes of assistance for the patient's arm movements. The MIB scenario was evaluated with 16 stroke patients, who rated it positively using the Intrinsic Motivation Inventory questionnaire. Additionally, the MIB scenario seems to elicit higher motivation than a simpler pick-and-place training task.

This paper presents the analysis of four psychophysiological responses in post-stroke upper extremity rehabilitation. The goal was to determine which psychophysiological responses would provide the most reliable information about subjects' psychological states during rehabilitation. Heart rate, skin conductance, respiration, and skin temperature were recorded in a stroke group and a control group during two difficulty levels of a pick-and-place task performed in a virtual environment using a haptic robot and during a cognitive task. Psychophysiological measurements were correlated with results of a self-report questionnaire. All four responses showed significant changes in response to the different tasks. Skin conductance differentiated between the two difficulty levels and was correlated with self-reported arousal in both stroke and control groups. Skin temperature differentiated between the two difficulty levels for the control group, but provided poor results for the stroke group. Heart rate and respiration increased during tasks, but their connection to psychological state was unclear. Results suggest that, of the four measured responses, skin conductance offers the most potential as a psychological state indicator, with other measures providing supplementary information. Psychophysiological measurements could thus be used in closed-loop biocooperative systems that would detect the user's psychological state and change the course of therapy accordingly.

This paper presents the novel "River" multimodal rehabilitation robotics scenario that includes video, audio and haptic modalities. Elements contributing to intrinsic motivation are carefully joined in the three modalities to increase motivation of the user. The user first needs to perform a motor action, then receives a cognitive challenge that is solved with adequate motor activity. Audio includes environmental sounds, music and spoken instructions or encouraging statements. Sounds and music were classified according to the arousal-valence space. The haptic modality can provide catching, grasping, tunnel or adaptive assistance, all depending on the user's needs. The scenario was evaluated in 16 stroke users, who responded to it favourably according to the Intrinsic Motivation Inventory questionnaire. Additionally, the river multimodal environment seems to elicit higher motivation than a simpler apple pick-and-place multimodal task.