Immersive techniques such as augmented reality through devices such as the AR-Sandbox and deep learning through convolutional neural networks (CNN) provide an environment that is potentially applicable for motor rehabilitation and early education. However, given the orientation towards the creation of topographic models and the form of representation of the AR-Sandbox, the classification of images is complicated by the amount of noise that is generated in each capture. For this reason, this research has the purpose of establishing a model of a CNN for the classification of geometric figures by optimizing hyperparameters using Random Search, evaluating the impact of the implementation of a previous phase of color-space segmentation to a set of tests captured from the AR-Sandbox, and evaluating this type of segmentation using similarity indexes such as Jaccard and Sørensen-Dice. The aim of the proposed scheme is to improve the identification and extraction of characteristics of the geometric figures. Using the proposed method, an average decrease of 39.45% to a function of loss and an increase of 14.83% on average in the percentage of correct answers is presented, concluding that the selected CNN model increased its performance by applying color-space segmentation in a phase that was prior to the prediction, given the nature of multiple pigmentation of the AR-Sandbox.
Fine motor skills allow to carry out the execution of crucial tasks in people's daily lives, increasing their independence and self-esteem. Among the alternatives for working these skills, immersive environments are found providing a set of elements arranged to have a haptic experience through gestural control devices. However, generally, these environments do not have a mechanism for evaluation and feedback of the exercise performed, which does not easily identify the objective's fulfillment. For this reason, this study aims to carry out a comparison of image recognition methods such as Convolutional Neural Network (CNN), K-Nearest Neighbor (K-NN), Support Vector Machine (SVM) and Decision Tree (DT), for the purpose of performing an evaluation and feedback of exercises. The assessment of the techniques is carried out using images captured from an immersive environment, calculating metrics such as confusion matrix, cross validation and classification report. As a result of this process, it was obtained that the CNN model has a better supported performance in 82.5% accuracy, showing an increase of 23.5% compared to SVM, 30% compared to K-NN and 25% compared to DT. Finally, it is concluded that in order to implement a method of evaluation and feedback in an immersive environment for academic training in the first school years, a low margin of error must be taken in the percentage of successes of the image recognition technique implemented, to ensure the proper development of these skills considering their great importance in childhood.
HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.