A New Knowledge Primitive of Digits Recognition for NAO Robot Using MNIST Dataset and CNN Algorithm for Children’s Visual Learning Enhancement

Abstract

Aim/Purpose: Our study is focused on prototyping, development, testing, and deployment of a new knowledge primitive for the humanoid robot assistant NAO, in order to enhance student visual learning by establishing a human-robot interaction. Background: This new primitive, utilizing a convolutional neural network (CNN), enables real-time recognition of handwritten digits captured by the NAO robot, a humanoid robot assistant developed by SoftBank Robotics. It is equipped with advanced capabilities, including a wide range of sensors, cameras, and interactive features. By integrating the proposed primitive, the NAO robot gains the ability to accurately recognize handwritten digits, contributing to improved student visual learning experiences. Methodology: Our developed primitive consists of the use of a convolutional neural network (CNN) so that the robot is able to recognize the handwriting of the digits present in the input image received in real-time. The NAO robot establishes interaction with the learners through a scenario based on a predefined assignment. In this scenario, NAO captures the digit handwritten by the learner via its camera, recognizes the digit using the deep learning model generated by the MNIST dataset, and announces to the learner the handwritten digit in the input image. The prototype is realized using the concept of a distributed system allowing the distribution of tasks in four different computing nodes. Contribution: Our research makes a significant contribution by equipping the humanoid robot NAO with a cognitive intelligence system through the integration of a new knowledge primitive based on handwriting digit recognition (HWDR). Our approach used to create and implement this primitive in the NAO robot is interesting and innovative, and presents a promising provision for enhancing the visual learning experience of children and young students with special needs, based on the use of distributed systems that divide the work using various components distributed over several nodes, coordinating their efforts to perform tasks more efficiently than a single device besides the NAO robot. Findings: We designed our model using specific parameters and a fully convolutional neural network architecture, which includes three residual depthwise separable convolutions, each followed by batch normalization and ReLU activation. To evaluate the performance of our model, we tested it on the MNIST dataset, where we achieved a remarkable accuracy, F1 score, and recall of 99%. An experiment was conducted to test our implemented primitive and see the effectiveness of this invention for enhancing visual learning in children with special needs. We developed a visual learning strategy based on the creation of engaging activities mediated by the NAO robot in an educational context. The results showed that participants achieved a strong commitment to the NAO robot, appreciating its ability to recognize handwritten digits and highlighting its promising potential to enrich visual learning experiences. Participants expressed a strong preference for teaching methods integrating assistive learning technologies, demonstrating the positive impact of our humanoid assistant robot on improving learning and visual intelligence in an educational environment. Recommendations for Practitioners: Encourage creativity and innovation in the field of robotics and special needs. This can lead to new and effective solutions that improve the lives of students with special needs. Recommendation for Researchers: Test and evaluate the proposed robotics solutions to ensure they are effective and making a positive impact. Use feedback from users, educators, and parents to refine and improve your solutions. Also, ensure that the robotics solutions are accessible to students with a range of abilities. This may involve designing solutions that are adjustable or providing alternative means of access. Impact on Society: As there are several ways to educate, there are multiple forms of learning. With the help of this learning procedure and strategy, the human teacher collaborates with the robot assistance NAO to improve visual learning among students. The findings of this research can serve as an application for the implementation of various pedagogical methods that will assist in meeting the needs of the majority of learners. Future Research: Our future research will concentrate on addressing the educational needs of students with special needs, enabling them to overcome their challenges and reach academic excellence in an inclusive environment. To achieve this goal, we plan to leverage the capabilities of social robots, which have emerged as a significant contributor to the field of human-robot interaction, particularly in facilitating inclusive education. These agents have proven to be effective in providing support to students with special needs, thereby enabling them to receive the education they need to succeed.