Effectiveness of AI in solving math problems at the secondary school level

Abstract

The research aims to evaluate the effectiveness of artificial intelligence (AI) in solving mathematical problems at the high school level by comparing the performance of AI with that of students. Utilizing the ChatGPT tool and a dataset of 20 mathematics questions from the high school curriculum, the study involved ten students from the Muhammadiyah Mertoyudan Islamic Boarding School. The results indicate that AI exhibits high accuracy in multiple-choice questions (98%) and short-answer questions (95%), albeit with a decrease in performance for essay questions (75%); conversely, students demonstrated an average accuracy of 85% for multiple-choice questions, 80% for short-answer questions, and 70% for essay questions, additionally, also demonstrated higher consistency, particularly in questions requiring complex conceptual understanding. At the same time, the findings underscore the significant potential of AI as a tool for mathematics learning, further development is needed to enhance its ability to comprehend and respond to essay questions, thereby improving conceptual understanding and critical reasoning. The implications of these findings can contribute to the advancement of more sophisticated and effective educational technology to support mathematics learning in schools.