Detection of bimanual gestures everywhere: Why it matters, what we need and what is missing

Abstract

Bimanual gestures are of the utmost importance for the study of motor coordination in humans and in everyday activities. A reliable detection of bimanual gestures in unconstrained environments is fundamental for their clinical study and to assess common activities of daily living. This paper investigates techniques for a reliable, unconstrained detection and classification of bimanual gestures. It assumes the availability of inertial data originating from the two hands/arms, builds upon a previously developed technique for gesture modelling based on Gaussian Mixture Modelling (GMM) and Gaussian Mixture Regression (GMR), and compares different modelling and classification techniques, which are based on a number of assumptions inspired by literature about how bimanual gestures are represented and modelled in the brain. Experiments show results related to 5 everyday bimanual activities, which have been selected on the basis of three main parameters: (not) constraining the two hands by a physical tool, (not) requiring a specific sequence of single-hand gestures, being recursive (or not). In the best performing combination of modelling approach and classification technique, five out of five activities are recognised up to an accuracy of 97%, a precision of 82% and a level of recall of 100%.

Gesture-based Human-Machine Interaction: Taxonomy, Problem Definition, and Analysis

“…In fact, if just a subset p(t) ⊆ q(t) is needed to represent a gesture, then the trajectory associated with that gesture can be redefined as τ p (t s , t e ). It is noteworthy that focus can also model gestures referring to multiple, or even non directly connected, body parts, e.g., bimanual gestures [49].…”

Performance Evaluation of Optical Motion Capture Sensors for Assembly Motion Capturing

“…Dynamic movement primitive based methods [26] utilize second-order dynamic systems to imitate the shape of motion trajectories. Gaussian mixture model based methods [7], [13], [14] learn the latent locations based on multiple demonstrated motion trajectories. Hence trajectory performance of optical MoCaps ought to be considered.…”

“…Acquiring human assembly motion is the first step in a typical LfD framework [6]. Accurate and smooth demonstration data are important for the study of LfD [7]. To this end, it is necessary to find a demonstration platform suitable to obtain the human operators' 3C product assembly motions, i.e.…”