Gesture Recognition Portfolios for Personalization

Abstract

Human gestures, similar to speech and handwriting, are often unique to the individual. Training a generic classifier applicable to everyone can be very difficult and as such, it has become a standard to use personalized classifiers in speech and handwriting recognition. In this paper, we address the problem of personalization in the context of gesture recognition, and propose a novel and extremely efficient way of doing personalization. Unlike conventional personalization methods which learn a single classifier that later gets adapted, our approach learns a set (portfolio) of classifiers during training, one of which is selected for each test subject based on the personalization data. We formulate classifier personalization as a selection problem and propose several algorithms to compute the set of candidate classifiers. Our experiments show that such an approach is much more efficient than adapting the classifier parameters but can still achieve comparable or better results.

Personalizing Gesture Recognition Using Hierarchical Bayesian Neural Networks

“…We observe that personalization using BALD outperforms personalization using RAND when the number of personalization instances is greater than 1 for the MSRC-12 dataset (left), 3 for the ChaLearn 2013 dataset (middle) and 4 for the NATOPS dataset (right). Our results also compare favorably with the personalization methods presented by Yao et al [34], who reported their results for the MSRC-12 and ChaLearn 2013 datasets. We compare the personalization results with a baseline BNN trained with all training data pooled into one group, whose mean is depicted in the figures as a dashed black line.…”

“…Like Yao et al [34], we experimented with the Training and Validation data containing ∼11000 samples. The gestures in the dataset, recorded using the Microsoft Kinect, represent common communication signals used in the Italian language (Figure 3 middle).…”

“…For controlled comparisons with previous work [34], we used identical feature representations -raw x,y,z world coordinates for 20 body joints in the MSRC-12 and Chalearn datasets. For NATOPS, we used the 20 dimensional features made available in [29], per frame.…”

“…A particular challenge faced by personalization systems is the small amounts of subject-specific data available for personalization. Yao et al [34] tackled this by recasting the problem into one of selecting the best performing model from a portfolio of pre-trained models. Since no new learning occurs, the approach is very data efficient.…”

“…Specifying them manually can be difficult and authors in the past [34] have resorted to setting them via cross-validation. Although cross-validation procedures can be effective for simpler models, they are untenable here.…”