An Assessment of Item Statistics Estimates of Basic Education Certificate Examination through Classical Test Theory and Item Response Theory approach

Abstract

In this study, the researchers assessed comparability of item statistics of 2017 basic education certific ate mathematics examination of National Examinations Council (NECO) through Classical Test Theory (CTT) and Item Response Theory (IRT) measurement frameworks. The study adopted instrumentation design. A 60-item NECO basic education certificate education mathematics objective test paper I was administered to 978 bas ic nine examinees, randomly selected from Osogbo and Olorunda Local Government Area, Osun S tate, Nigeria. The responses of the examinees to the test data were analysed using Marginal Maximum Likelihood Estimation of JMETRIK software. The result showed that the test data obey the assumption of unidimensionality of 3parameter logistic model and Classical Test Theory measurement framework deleted more items 33 (55%) compare to IRT measurement framework 12 (20%). Also, it was observed that item statistics fro m the two contrasting frameworks (CTT and IRT) were not comparable. Moreover, further analysis showed that there was low c orrelation among the item statistics index. The implication of this is that NECO should jettison the use of Classical Test Theory and embrace utilization of Item Response Theory framework during their test development and item analysis.

The use of measurement frameworks to explore the qualities of test items

“…Furthermore, when comparing the quality of the test items CTT accepted 11 (36%) items as good items whereas IRT (3PLM) accepted 23 (77%) items and in terms of difficulty and discriminating indices, the CTT and IRT parameter estimates differed significantly. This is in line with the findings of [19], who discovered that item parameter estimates for CTT and IRT basic education certificates examinations were not comparable. This finding support the research [9] which found a disparity between CTT and IRT and explained further that in terms of very difficult items and the most appropriate model for data assessment is IRT.…”

“…Contrariwise, when comparing and testing for significance differences between item parameter estimates from CTT and those from 2-3-IRT across models for inter and intra model validation, there was no significant inter validation of item parameter estimates between CTT and 2-3-IRT models [29]. In support of previous studies [19], [29] from their study of basic education certificate examination, analysis indicate that CTT and IRT item statistics were just not comparable. It was indicated further that CTT deleted so much items than the IRT measurement framework.…”

“…These two theories are doubtless the major frameworks employed construct, assess, and analyze test items in educational measurements. Different [19].…”

“…Furthermore, as Zóltak and Golonka [41] pointed out that determining the IRT model fit is critical when drawing conclusions from any data. Assessing the model data fit of EDU222 considering the submission of previous study [38], [43], the best fit is provided by the model with the lowest AIC, BIC, and -2 likelihood ratio value. Thus, the finding shows that the 3-parameter logistic model provides the best fit to the data, enabling tendency for guessing.…”

Calibration of Polytomous Response Mathematics Achievement Test Using Generalized Partial Credit Model of Item Response Theory

“…(2) Be that as it may, [19,21] pointed out that bj -dj,k is the point on the "q" scale at which the plots of Pj,k-1(q) and Pjk(q) meet, so characterise the point on the "q" scale at which the response to item "j" has risen Survey of literature has shown that in Nigeria, down to earth applications of IRT models for item calibration has ibeen ioverwhelmingly iutilised iunder imultiple-choice iitems iand ipublic iexamining ibodies iin isubsahara iAfrica isuch ias iNECO, iWAEC ietc iwere inot iexempted ifrom ithis itrend, iwhile iless iattention iwas ipaid ito icalibration iof ipolytomous itest iitems, iwhich icomplement ithe imultiple-choice iitems iused iby ithe iexamination ibodies ifor itheir ihigh-stake itesting. iFor ii nstance, [5,[22][23][24][25][26][27][28][29][30] worked broadly on calibrations of multiplechoice items of different subjects. Whereas researchers [18,19,[31][32][33] in developed nations like USA, UK, Germany, Ireland etc have grasped this approach of utilising IRT application for the foundation of constructed-response test items (polytomous) parameters.…”