Using Confusion Reject to Improve (User and) System (Cross) Learning of Gesture Commands

Abstract

This paper presents a new method to help users defining personalized gesture commands (on pen-based devices) that maximize recognition performance from the classifier. The use of gesture commands give rise to a cross-learning situation where the user has to learn and memorize the command gestures and the classifier has to learn and recognize drawn gestures. The classification task associated with the use of customized gesture commands is complex because the classifier only has very few samples per class to start learning from. We thus need an evolving recognition system that can start from scratch or very few data samples and that will learn incrementally to achieve good performance after some using time. Our objective is to make the user aware of the recognizer difficulties during the definition of commands, by detecting confusion among gesture classes, in order to help him define a gesture set that yield good recognition performance from the beginning. To detect confusing classes we apply confusion reject principles to our evolving recognizer, which is based on a first order fuzzy inference system. A realistic experiment has been made on 55 persons to validate our confusion detection technique, and it shows that our method leads to a significant improvement of the classifier recognition performance.

User and System Cross-Learning of Gesture Commands on Pen-Based Devices

“…It allows us to flag gestures as confusing when confidence is below a defined confidence threshold (0.09 in our case). The choice of the threshold is explained in our paper [17].…”

Online active supervision of an evolving classifier for customized-gesture-command learning

“…On the other hand, enabling users to choose their own gestures may lead to commands with similar or strange gestures that are hard to recognize by the classifier. We must assist the user to avoid similar gestures during this definition step, by providing a dynamic feedback on potential confusion risks [5]. Moreover, we can't expect users to draw much more than a few gesture samples per class, so the classifier must be able to learn with very few data.…”

Online handwriting recognition using depth sensors

“…These devices rarely use a mouse or keyboard and employs natural interfaces like gesture and speech instead. Natural interfaces using depth sensors like kinect are getting popular for gesture recognition [1,2,3,4], sign language recognition [5], signature based authentication systems [6], and computer games. Wachs et al [7] also talks about vision based hand-gesture applications, which include medical systems and assistive technologies, crisis management and disaster relief, entertainment and human-robot interaction.…”