Transforming Hand Gesture Recognition Into Image Classification Using Data Level Fusion

Abstract

Hand gesture recognition (HGR) is a form of perceptual computing with applications in human-machine interaction, virtual/augmented reality, and human behavior analysis. Within the HGR domain, several frameworks have been developed with different combinations of input modalities and neural network architectures to varying levels of efficacy. Such frameworks maximized performance at the expense of increased computational and hardware requirements. These drawbacks can be mitigated by a skeleton-based framework that transforms the hand gesture recognition task into an image classification task. This chapter explores several temporal information condensation (via data-level fusion) methods for encoding dynamic gesture information into static images. The efficacies of these methods are compared, and the best ones are aggregated into a generalized HGR framework which was extensively evaluated on the CNR, FPHA, LMDHG, SHREC2017, and DHG1428 benchmark datasets. The framework's performance shows competitiveness compared to other frameworks within the state-of-the-art for the datasets.