Dimensionality assessment of binary response test items: A non-parametric approach of bayesian item response theory measurement

Abstract

This study investigated dimensionality of Binary Response Items through a non-parametric technique of Item Response Theory measurement framework. The study used causal comparative research type of non-experimental design. The sample consisted of 5,076 public senior secondary school examinees (SSS3) between the age of 14-16 years from 45 schools, which were drawn randomly from three senatorial districts of Osun State, Nigeria. Instrument used for this study was 2018 Osun State unified multiple-choice mathematics achievement test items with empirical reliability coefficient of 0.82. Data obtained were analysed using Non-linear factor analysis, Stout’s Test of Essential Unidimensionality (STEU), Factor Analysis (FA), Full Information Factor Analysis (FIFA) and Bootstrap Modified Parallel Analysis Test (BMPAT). Results showed that both the BMPAT and STEU ascertained violation of unidimensionality assumption of the test items (the observed difference in the second eigenvalue of the observed data and that of second eigenvalue of the simulated data was statistically significant, p = 0.0099; Stout’s test rejected the assumption of essential unidimensionality, T = 10.6260, p<0.05). Non-linear factor analysis and full information factor analysis revealed that four dimensions embedded in the test items and loadings of the items showed within-item multidimensionality respectively. The authors’ concluded that modeling examinees’ performance with unidimensional model when it was actually multidimensional in nature would affect performance of examinees adversely and could lead to blur judgment. Consequently, it is recommended that unidimensional scoring method of Osun State unified mathematics achievement test implicit in Classical Test Theory should be jettisoned and an appropriate scoring model (multidimensional) should be embraced.

Performance of exploratory structural equation model (ESEM) in detecting differential item functioning

“…For instance, the National Benchmark Test in South Africa consists of Academic Literacy, Quantitative Literacy, and Mathematics; Graduate Management Admission Test consists of mathematics, verbal reasoning, quantitative reasoning, and English language; Joint Admission and Matriculation Board consist of Mathematics, English, Physics, Chemistry, Biology and so on. Also, this result laid credence to the findings of [5,6], which argued that there was no evidence of unidimensionality in 2018 Osun State unified multiple-choice mathematics achievement test items. However, this result was in dissonance with the findings of [4,48] that the unidimensionality of the test was met when comparing three methods for evaluating dimensionality.…”

The use of measurement frameworks to explore the qualities of test items

“…Uni-dimensionality is a key feature of measurement theory that can be used to evaluate variable interrelationships. When the eigenvalue of the first value has higher percentage over the next one, uni-dimensionality is usually supported [37], [42]. The finding revealed that EDU222 achievement test sufficiently meets the assumption of unidimensionality, as determined by [33] using factor analysis with the eigenvalues-greater-than-one rule.…”

Implementation of Web-based Character Assessment on Students' Character Outcomes: A Review on Perception and Gender

“…This evaluation activity is generally carried out conventionally (Barua, Singh, Mridiula, Gupta, Satyapriya & Tomar, 2017;Hardianti,Taufiq & Pamelasari, 2017;Abdurrahman, Saregar & Umam, 2018;Ozan, 2019;Violato & King, 2021). However, with the development of technology, evaluation can be integrated with technology (Sakar, Orhan, Sinanoğlu, Tosun & Öz, 2017;Ayanwale, Isaac-Oloniyo & Abayomi, 2020;Esomonu, Esomonu & Eleje, 2020). So it is necessary to develop a technology-based assessment system to facilitate the assessment process.…”