Deep Learning Based Machine Vision: First Steps Towards a Hand Gesture Recognition Set Up for Collaborative Robots

Deep learning-based hand gesture recognition for collaborative robots

“…The results are shown in Table 1, for the five datasets. Thanks to the modified training strategy and the random shuffle with a different seed, the training times obtained in this work are lower with respect to the ones obtained in [4] (reported in the right column of the Table), with the only exceptions of the Zoom and of the Complete datasets. This is by no means surprising, since the number of examples in the two training datasets increased by 53% and by 11% with respect to the first implementation, respectively.…”

“…In a previous work, detailed in [4], we presented a system based on Deep Learning for the recognition of gestures based on the simultaneous presence of the left hand closed as the anchor gesture and of the right hand used to specify the type of gesture, the focus being on the robustness of the recognition, even at the expense of gesture naturalness. Suitable datasets of images were collected and a Faster Region Proposal Convolutional Neural Network (or Faster R-CNN) [5] was implemented to detect the gestures.…”

“…2, our complete procedure is composed of two main blocks: the Faster R-CNN Object Detector block and the Custom Prediction Function (CPF) block. The Faster R-CNN detector carries out the detection of single-hand gesture; since it is the same as the one detailed in [4], [5], it is not presented in this paper. For the sake of clarity, it is sufficient to mention that its objective is to provide a set of predictions to the CPF block.…”

“…◗ The four datasets were randomly shuffled separately, to ensure a random selection of the data in the following step; ◗ From each dataset, 80% of the data has been selected for training (Complete training dataset) and the remaining 20% from each dataset has been selected for testing (Complete test dataset); ◗ The two datasets obtained so far (training and test) were both randomly shuffled, to ensure that the data in every batch used by the algorithm for training was not all from the same original dataset (Base, Light Colors, Gloves or Zoom). This strategy was adopted to overcome the limitations of the one used in [4], where the Complete training and test datasets have been obtained through the following steps:…”

“…◗ The four datasets were combined into one dataset (Complete total dataset); ◗ The Complete total dataset was randomly shuffled; ◗ From the Complete total dataset, 80% of the data was selected for training (Complete training dataset) and the remaining 20% of the data was selected for testing (Complete test dataset). In [4], the latter strategy turned out to be only partially adequate, the reason being that combining the four datasets first and then performing a random shuffle of the Complete total dataset cannot guarantee that the training and test datasets are going to be composed of a proportional number of samples from each original dataset.…”