Towards an Automatic Annotation of French Sign Language Videos: Detection of Lexical Signs

Abstract

This paper presents an approach towards an automatic annotation system for French Sign Language (LSF). Such automation aims to reduce the processing time and the subjectivity of manual annotations done by linguists in order to study the sign language and simplify indexing for automatic signs recognition. The described system uses face and body keypoints collected from 2D RGB standard LSF videos. A naive Bayesian model was built to classify gestural units using the collected keypoints as features. We started from the observation that, for many signers, the production of lexical signs is very often accompanied by mouthing. Effectively, the results showed that the system is capable of detecting lexical signs, with highest success rate, using only information about mouthing and head direction.