Towards a platform-independent cooperative human-robot interaction system: II. Perception, execution and imitation of goal directed actions

Abstract

If robots are to cooperate with humans in an increasingly human-like manner, then significant progress must be made in their abilities to observe and learn to perform novel goal directed actions in a flexible and adaptive manner. The current research addresses this challenge. In CHRIS.I [1], we developed a platform-independent perceptual system that learns from observation to recognize human actions in a way which abstracted from the specifics of the robotic platform, learning actions including ?????????put X on Y????????? and ?????????take X?????????. In the current research, we extend this system from action perception to execution, consistent with current developmental research in human understanding of goal directed action and teleological reasoning. We demonstrate the platform independence with experiments on three different robots. In Experiments 1 and 2 we complete our previous study of perception of actions ?????????put????????? and ?????????take????????? demonstrating how the system learns to execute these same actions, along with new related actions ?????????cover????????? and ?????????uncover????????? based on the composition of action primitives ?????????grasp X????????? and ?????????release X at Y?????????. Significantly, these compositional action execution specifications learned on one iCub robot are then executed on another, based on the abstraction layer of motor primitives. Experiment 3 further validates the platform-independence of the system, as a new action that is learned on the iCub in Lyon is then executed on the Jido robot in Toulouse. In Experiment 4 we extended the definition of action perception to include the notion of agency, again inspired by developmental studies of agency attribution, exploiting the Kinect motion capture system for tracking human motion. Finally in Experiment 5 we demonstrate how the combined representation of action in terms of perception and execution provides the basis for imitation. This provides the basis for a- open ended cooperation capability where new actions can be learned and integrated into shared plans for cooperation. Part of the novelty of this research is the robots' use of spoken language understanding and visual perception to generate action representations in a platform independent manner based on physical state changes. This provides a flexible capability for goal-directed action imitation

Compliance Control and Human–Robot Interaction: Part II — Experimental Examples

“…Note that the adaptation laws of (18)- (21) are also including c in contrast to (7)- (10). This modifies for c → +0 the adaptive laws into autonomous stable systems so that windup prevention is introduced, which is enhanced by the increase of the forgetting factor in (22). Using this modified adaptive law, this changes the control law to: …”

“…The system consists of (i) an anthropomorphic hand, comprising capabilities like grasping, pointing, releasing and 'sensitive' hand-over 22,23 (see section 4.2); (ii) a humanoid robot arm fitted with a model reference adaptive compliance controller with contact detection, running on a dSPACE system; (iii) a bi-directional spoken language interface to issue commands, report states and request instructions 22,23 (see Section 4.3). We show how those subsystems, all running on different computational platforms, can be integrated, and demonstrate the system's functionality with an interaction scenario that utilises all (built in) modalities.…”

“…In our scenario, two locations (i.e. 'close to human' and 'above bin') are pre-programmed, but could easily be dynamically set or acquired by a higher level perception and reasoning system, as reported by Lallee et al 22 . The human can now verbally instruct the robot to move to him/her or to the bin.…”

“…For the grasping of objects, a single anthropomorphic hand module ( Figure 6) has been integrated into the arm control infrastructure 22,23,17 . The hand has nine DOF comprising gross movement for all 5 fingers, "trigger" action for index and middle finger, opposing of the thumb, and finger spread.…”

“…In this case, the adaptation laws (7)-(10) are disabled avoiding windup. The increased forgetting factor of (22) guarantees that the adaptive control law decreases its amplitude so that normal operation is recovered. The adaptive compliance controller with posture torque controller and AW compensator given by (14) and (18)- (24) is locally stable and achieves an ultimately bounded tracking error.…”