Assessing Students’ Higher Order Thinking Skills Using Multidimensional Item Response Theory

Abstract

The development of the economics HOTS test that combines critical thinking skills, problem-solving, and critical thinking are essential to meet the challenges of the 21 st century life skills. The combination of these thinking skills in the HOT test will help teachers to diagnose students' strengths and weaknesses. However, it could interfere with the accuracy of the measurement results if analyzed using IRT in a single analysis. The Multidimensional Item Response Theory (MIRT) resolved the accuracy of the measurement issues. This research aimed to develop the economics HOTS test to estimate the student's abilities in higher order thinking skills using the MIRT. The samples were 750 high school students selected from fourteen high schools in West Sumatera, Indonesia. The data were collected using tests which calibrated through the simple-structure MIRT model using R studio. The test reliability was calculated based on coefficients Alpha and test information function. The results show MIRT offers accurate measurement in estimating multidimensional test parameters. The item had a moderate average multidimensional discriminant and difficulty, while the students had a moderate HOTS ability. Their ability to think creatively was lower than critical thinking and problem-solving abilities. The test was proven to be reliable with coefficients Alpha of 0.81, it yielded a high-test information function of the 4.0124, and a low measurement error of 0.4992. It is suitable to be tested on students who have moderate abilities in problem-solving and critical thinking, but with high creative thinking ability.

HOTS in Quadratic Equations: Teaching Style Preferences and Challenges Faced by Malaysian Teachers

“…However, a study by Irma-Ain and Zolkepeli (2017) have found that mathematics teachers in Malaysia are less skilled in applying strategies for the usage and learning of higher-order thinking skills in teaching and learning. The finding is supported by Friyatmi (2020), who has reported that most teachers are incapable of formulating higher-order thinking skills related questions and items. The difficulties faced by the teachers in constructing higher-order questions show that the challenge of cultivating higher-order thinking skills within the teaching of mathematics not only arises from the aspect of pedagogical skills, but also insufficient existing knowledge among the teachers.…”

Item parameter estimations for multidimensional graded response model under complex structures

“…We identified 60 articles across various contexts that applied MGRM with empirical data using databases (e.g., Google Scholar and Worldwide Science). Among the identified articles, we noted that MGRM has been gaining attention from researchers in health (e.g., Walton et al, 2008;Depaoli et al, 2018;Haem andDoostfatemeh, 2020) andeducation (e.g., DeMars, 2013;Wang et al, 2015;Friyatmi, 2020). These studies used MGRM because it improved measurement precision, especially when the correlation between latent traits was taken into account, and the test lengths and sample sizes were modest.…”

“…The application of MGRM has occurred in both education (Wang et al, 2004;Friyatmi, 2020) and health-related contexts (e.g., mental health;DeMars, 2013;Nouri et al, 2021). Using GRM (or other multidimensional polytomous IRT models) to describe the interaction of items and examinees necessitates the specification of either a simple or complex structure.…”

Development of Mobile Learning Media (MLM) to Enchance Students’ Understanding of CNC Programming Subjects

“…There have been many strategies to use technology to develop student competencies [17]- [20]. Mobile learning is a form of using technology in learning [15], [21]- [23].…”