Sensory-Motor Manifold Structure Induced by Task Outcome: Experiments with Robonaut

Abstract

Abstract-Experiments were performed with Robonaut to determine if repeated teleoperation through a task could delineate a manifold in the robot's sensory-motor state-space (SMSS) that was separable by the outcome of the task. A teleoperator guided the robot through 13 trials of a pick and place task. By design, in 5 of the trials were successful and 8 were not. A time series of instantaneous sensory-motor state vectors was recorded at 8Hz from the robot throughout the trials. Support Vector Machine (SVM) analysis suggested that the individual vectors were separable with a 70% probability if both sensory and motor information were used. The analysis revealed, however, little structural information beyond the promise of reasonable separability. Therefore, dimensionality reduction techniques were applied. These embed high-dimensional vectors into a lower dimensional space where patterns may be discernible. Singular Value Decomposition, Multidimensional Scaling, and SpatioTemporal Isomap (STI) analysis of the time series revealed a 3D structure within the SMSS that was dependent on, and separable by task outcome. An STI embedding generated by 2 successful and 4 unsuccessful trials was seen to be sufficient for the projection and classification of the remaining trials. A comparison of that result with the SVM classification demonstrated that STI outperformed SVM. It is conjectured that STI's reliance on the temporal evolution of the time-series gave it the advantage over the SVM analysis which was time independent. It was also found that Robonaut's haptic sensory data degraded the outcome separability if used without processing. However, lateral inhibition of the haptic signals significantly enhanced separability in both the SVM and STI analyses.

Interactive object recognition using proprioceptive and auditory feedback

“…First, other methods for dimensionality reduction (e.g. vector quantization, or spatio-temporal isomaps, as used in Peters et al (2006)) can be applied in order to find meaningful patterns in the robot's proprioceptive and auditory sensory streams. Second, while the robot in our study was tested on an object recognition task, it is also possible to use auditory and proprioceptive feedback to detect certain physical properties of the object (e.g.…”

Online motion selection for semi-optimal stabilization using reverse-time tree

“…Peters et al [11] proposed a method to discriminate successful from unsuccessful trials from sensor recordings of a teleoperated robot. Pastor et al [4] proposed a method to use sensor recordings from previous trials to early detect failure conditions in future trials.…”

An information pipeline model of human-robot interaction

“…Though Robonaut's input devices include wearable gloves and position trackers for the arms, neck, and waist of the operator, in practice the human operator cannot effectively control such a high-information signal in real-time. As a result, all three strategies mentioned above are used to augment the information content of the human's signal: primitives for hand motions reduce the space of possible motor actions of the robot's hand; the human can issue verbal "freeze" and "thaw" commands to various joints so that the robot parts can be controlled in serial over time; and work continues on finding ways to extract the necessary information for motor tasks from the sensory and motor inputs [23].…”