Predicting Intentions from Motion: The Subject-Adversarial Adaptation Approach

Abstract

This paper aims at investigating the action prediction problem from a pure kinematic perspective. Specifically, we address the problem of recognizing future actions, indeed human intentions, underlying a same initial (and apparently unrelated) motor act. This study is inspired by neuroscientific findings asserting that motor acts at the very onset are embedding information about the intention with which are performed, even when different intentions originate from a same class of movements. To demonstrate this claim in computational and empirical terms, we designed an ad hoc experiment and built a new 3D and 2D dataset where, in both training and testing, we analyze a same class of grasping movements underlying different intentions. We investigate how much the intention discriminants generalize across subjects, discovering that each subject tends to affect the prediction by his/her own bias. Inspired by the domain adaptation problem, we propose to interpret each subject as a domain, leading to a novel subject adversarial paradigm. The proposed approach favorably copes with our new problem, boosting the considered baseline features encoding 2D and 3D information and which do not exploit the subject information. Keywords Action recognition and prediction • Human intentions • Grasping • Kinematic analysis • Adversarial domain adaptation 1 Introduction Recognizing human actions is an active area of research which is faced under different paradigms in computer vision and pattern recognition. The most straightforward one, Communicated by Kristen Grauman. Andrea Zunino and Jacopo Cavazza have contributed equally to this work.

Intra-Subject and Inter-Subject Movement Variability Quantified with Muscle Synergies in Upper-Limb Reaching Movements

“…Many studies showed that surface electromyography (EMG) signals contain rich information that could be exploited in many fields. They can be used to decode motion intention for recognizing grasping movements [ 4 ] or predicting gait events [ 5 ], and, especially, to understand the neurophysiological mechanisms of motor control [ 6 ]. It was found that the intra-subject variability is crucial in applications involving patients to track the evolution of the disease for diagnosing and treatment planning [ 7 , 8 ] or to assess healthy people’s performance as a reference for motor function evaluation [ 9 ].…”

A neural integrator model for planning and value-based decision making of a robotics assistant

“…Ideally, the robot should be able to inform the operator even before she has started to grasps the pipe. In many circumstances, humans are able to infer the motor intention of ongoing goal-directed movements solely based on the observed kinematics of arm and hand [65]. We are currently exploring the application of Deep Neural Networks (DNNs) [38] as a data driven approach to human action recognition and understanding in a manufacturing environment.…”

“…These representations may then be used to implement hierarchical processes that predict sensory events. Recent experiments show that DNN architectures may outperform humans in such complex classification tasks like predicting different final intentions of the same class of grasping movements (e.g., grasping for placing or grasping for passing a certain object, [65]). For typical HRI applications, the type of the part or the tool the operator is going to manipulate may help the intention classification (e.g., grasping a screw driver [60]).…”

Elbow Motion Trajectory Prediction Using a Multi-Modal Wearable System: A Comparative Analysis of Machine Learning Techniques

“…They can be used in ON/OFF systems, where a certain movement triggers a control routine to follow a pre-defined trajectory [ 27 ], or use a state-space model to decode the angular velocity of shoulder and motion of elbow [ 28 ]. One can also use position data from the initial portion of a movement to classify motion intention [ 29 ]. Kinematic data can be useful in motion intention detection if the user still has healthy function of their limbs.…”