Customizing skills for assistive robotic manipulators, an inverse reinforcement learning approach with error-related potentials

Abstract

Robotic assistance via motorized robotic arm manipulators can be of valuable assistance to individuals with upper-limb motor disabilities. Brain-computer interfaces (BCI) offer an intuitive means to control such assistive robotic manipulators. However, BCI performance may vary due to the non-stationary nature of the electroencephalogram (EEG) signals. It, hence, cannot be used safely for controlling tasks where errors may be detrimental to the user. Avoiding obstacles is one such task. As there exist many techniques to avoid obstacles in robotics, we propose to give the control to the robot to avoid obstacles and to leave to the user the choice of the robot behavior to do so a matter of personal preference as some users may be more daring while others more careful. We enable the users to train the robot controller to adapt its way to approach obstacles relying on BCI that detects error-related potentials (ErrP), indicative of the user’s error expectation of the robot’s current strategy to meet their preferences. Gaussian process-based inverse reinforcement learning, in combination with the ErrP-BCI, infers the user’s preference and updates the obstacle avoidance controller so as to generate personalized robot trajectories. We validate the approach in experiments with thirteen able-bodied subjects using a robotic arm that picks up, places and avoids real-life objects. Results show that the algorithm can learn user’s preference and adapt the robot behavior rapidly using less than five demonstrations not necessarily optimal.

EEG error-related potentials encode magnitude of errors and individual perceptual thresholds

“…2c). Although we have further analyzed datasets collected during our previous studies (27,29), we did not observe such gamma band activity nor theta-gamma phase-amplitude coupling. A distinction between those previous studies and the present study is that in the latter subjects performed corrective actions upon perception of an error.…”

“…Some BCIs have succeeded in decoding the presence of an ErrP during a continuous task (24)(25)(26)(27). However, these studies used experimental protocols where erroneous actions are abrupt stops of the device (24,25), errors happen at the same moment during the interaction (26,28), or the device fails to reach a target (27)-all extreme conditions that do not capture erroneous actions during a continuous interaction, but see Batzianoulis et al (29). Furthermore, none of these previous studies characterized the ErrP signals as a function of the severity of the erroneous action.…”

Brain computer interface to distinguish between self and other related errors in human agent collaboration

“…However, independently on the source of error that the classifier will be trained, it would be more sensitive to ErrPs with higher component amplitudes. We suggest that future co-adaptation 13 and shared-control BCI systems 15,53 may study the possible benefits from the detection of different self-and agent-related ErrPs. We suggest that, for subjects that have distinct self-and agent-related ErrPs, detecting the self-related ErrPs would improve the reliability (by correcting the misinterpreted user intention) and accuracy (by updating the user intentions decoder through time), and the detection of agent ErrPs would allow the system to better satisfy the user needs.…”

“…The ErrPs are characterized by distinctive positive and negative deflections (called components) in the recorded signals on the frontocentral EEG electrodes. Such passive BCI have already been suggested in the literature 7,8 and used to give neural feedback to computer interfaces 9,10 and robots [11][12][13][14][15] for ErrPs-based adaptation and co-adaptation. the human subject and the programmed autonomous agent worked in a shared workspace with a shared task 38 simulating assembly task where the order of the steps is preassigned.…”

A Massage Area Positioning Algorithm for Intelligent Massage System

“…In addition, Liu et al [ 18 ] proposed a multitask model combining STGCN-LSTM and YOLO to recognize human intentions. Batzianoulis et al [ 19 ] proposed the idea of determining control attribution based on people's personal preferences. Kim et al [ 20 ] have proposed a method to identify patterns in people's daily lives that combines intention and event algorithms.…”