Metacognitive Skills Assessment in Research-Proposal Writing (MSARPW) in the Indonesian University Context: Scale Development and Validation Using Multidimensional Item Response Models

Abstract

This study aimed to develop the Metacognitive Skills Assessment in a Research Proposal Writing Context (MSARPW), a multidimensional measure, based on student perspectives, for assessing metacognitive skills in thesis writing, and assess the instrument’s psychometric properties using item factor analysis (IFA) and multidimensional item response models. The 40-item MSARPW was administered to 602 Indonesian university students (Mage = 25.254 SDage = 6.854). The IFA showed that the two-dimensional factor structure of MSARPW was satisfactory; however, only 24 of the 40 items were found to fit the model. Multidimensional graded response models (MGRM) were applied to the subsequent 24-item MSARPW, which showed that one item (Item 4) did not satisfy the criteria. The estimated reliabilities of each subscale showed that the 23-item MSARPW has good internal consistency (0.891 and 0.902). To conclude, the 23-item MSARPW appears to be a valid and reliable tool for assessing metacognitive skills in the context of research-proposal writing among Indonesian university students.

Rasch Calibration and Differential Item Functioning (DIF) Analysis of the Indonesian National Assessment Program-Language (INAP-L)

“…The use of subscripts on m (that is, mj) indicates that the number of categories can vary for item to item. However, to have the unique characteristics of the Rasch model, certain requirements must be met, namely (Hayat et al, 2023;Yu, 2020): (1) unidimensionality, the instrument measures only one trait. The assumption of unidimensionality was tested in this study using a parallel analysis method based on minimum rank factor analysis (PA-MRFA; Timmerman & Lorenzo-Seva, 2011); and (2) local independence, which means that the test taker's response to an item must be independent of other items or persons.…”