Early Recognition of Handwritten Gestures Based on Multi-Classifier Reject Option

Abstract

In this paper a multi-classifier method for early recognition of handwritten gesture is presented. Unlike the other works which study the early recognition problem related to the time, we propose to make the recognition according to the quantity of incremental drawing of handwritten gestures. We train a segment length based multi-classifier for the task of recognizing the handwritten touch gesture as early as possible. To deal with potential similar parts at the beginning of different gestures, we introduce a reject option to postpone the decision until ambiguity persists. We report results on two freely available datasets: MGSet and ILG. These results demonstrate the improvement we obtained by using the proposed reject option for the early recognition of handwritten gestures.

Online Spatio-temporal 3D Convolutional Neural Network for Early Recognition of Handwritten Gestures

“…With this, the network needs to reject until the trajectory become discriminative, and it can be very late. We compared our score to the approach of Chen et al [4] using the predefined Train/Test split furnished by the dataset. To compare fairly with them, we used the parameter t which is the number of time the same prediction class must be accepted consecutively by our reject system to be finally accepted.…”

“…The MTGSetB dataset is composed of 45 different multi-touch gestures regrouped into 31 rotation invariant gesture classes made by 33 users. Like ILGDB, we compared our score to Chen et al [4]. 50% of the data are used for training.…”

“…t − 1 frame classifiers. Chen et al [4] addressed the task by using a combination of length-dependent classifiers and a system of reject based on the confidence scores of the classifiers and repetition of prediction. Recently, Yamagata et al [20] proposed an approach to do handwriting prediction which learns the bifurcations of gestures based on an LSTM network.…”

“…For the quick gesture, we would not be able to recognize the gesture as soon (in terms of quantity of information) as we would be able to recognize. To tackle these problems we used, as previous studies [2,4], the quantity of information (displacement) instead of the time to quantify the effective displacement and generate a speed independent image sequence. Each new image will be incremented by the same total displacement θ.…”

“…These two datasets are complementary in terms of gesture natures (mono/multi-stroke, mono/multi-touch), they are very interesting for early recognition experiments. We compare our scores to the state-of-the-art method on the task of early recognition [4] on these two datasets.…”