Calibration of Polytomous Response Mathematics Achievement Test Using Generalized Partial Credit Model

Ayanwale, Musa Adekunle

doi:10.2139/ssrn.3824911

Cited by 4 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…IRT is a set of algorithms that measure each item's characteristic on a scale, which corresponds to an individual's traits (Ayanwale, 2019;Ayanwale et al, 2022;van der Linden & Glas, 2010). In IRT models such as one-parameter (which looks at the difficulty -b of the item), two-parameter (which looks at the discrimination-a after item difficulty parameter is computed), three-parameter (which looks at the guessing-c in addition to b and a), and four-parameters (which looks at the carelessness-d in addition to b, a and c) the examinee's behaviour is taken into account at the item level (Ayanwale et al, 2018(Ayanwale et al, , 2019Baker, 2004). By modelling at the item level, scores can be reported, and CAT can be developed more efficiently.…”

Section: Literature Reviewmentioning

confidence: 99%

The feasibility of computerized adaptive testing of the national benchmark test: A simulation study

Ayanwale,

Ndlovu

2024

JPR

View full text Add to dashboard Cite

The COVID-19 pandemic has had a significant impact on high-stakes testing, including the national benchmark tests in South Africa. Current linear testing formats have been criticized for their limitations, leading to a shift towards Computerized Adaptive Testing [CAT]. Assessments with CAT are more precise and take less time. Evaluation of CAT programs requires simulation studies. To assess the feasibility of implementing CAT in NBTs, SimulCAT, a simulation tool, was utilized. The SimulCAT simulation involved creating 10,000 examinees with a normal distribution characterized by a mean of 0 and a standard deviation of 1. A pool of 500 test items was employed, and specific parameters were established for the item selection algorithm, CAT administration rules, item exposure control, and termination criteria. The termination criteria required a standard error of less than 0.35 to ensure accurate abilities estimation. The findings from the simulation study demonstrated that fixed-length tests provided higher testing precision without any systematic error, as indicated by measurement statistics like CBIAS, CMAE, and CRMSE. However, fixed-length tests exhibited a higher item exposure rate, which could be mitigated by selecting items with fewer dependencies on specific item parameters (a-parameters). On the other hand, variable-length tests demonstrated increased redundancy. Based on these results, CAT is recommended as an alternative approach for conducting NBTs due to its capability to accurately measure individual abilities and reduce the testing duration. For high-stakes assessments like the NBTs, fixed-length tests are preferred as they offer superior testing precision while minimizing item exposure rates.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

The feasibility of computerized adaptive testing of the national benchmark test: A simulation study

Ayanwale,

Ndlovu

2024

JPR

View full text Add to dashboard Cite

show abstract

“…Item response theory makes use of the pattern of replies to all the test items to simulate a test taker's skill and the likelihood that they would answer items correctly. Rather than the raw test results, the focus of IRT is on an examinee's accuracy on a given item (Ayanwale, 2019). Based on the pattern of item responses, the item-pattern scoring method yields a maximum probability trait estimate (Eaton, et al, 2019;Bichi & Talib, 2018).…”

Section: Introductionmentioning

confidence: 99%

Psychometric Assessment of Botswana's 2018 Junior Certificate Agriculture Science Items through Item Response Theory

Bulala,

Anane,

Cobbinah

2024

EAJESS

View full text Add to dashboard Cite

This study aimed to assess the psychometric properties of the Junior Certificate Examination (JCE) agricultural science multiple-choice items in Botswana, using the 3-parameter logistic model (3PLM). Adopting a quantitative descriptive research design, the study analyzed responses from 40,343 examinees who took the 40 multiple-choice items in the 2018 JCE agricultural science examination. The study utilized secondary data from Botswana Examination Council (BEC) records. They also obtained the ethical clearance from the Institutional Review Board of the University Cape Coast. The analysis, conducted using 3PLM within the R-studio environment revealed that 15% of items exhibited poor discrimination indices, 10% had difficulty and 12.5% were prone to guessing. There was an identification of Key anomalies in specific items, warranting careful consideration. Approximately 90% of the multiple-choice items fell within the good difficulty level range but 10% appeared to be poor difficulty. For item discrimination, up to 85% were within the acceptable range while 15% were outside. The study recommends a thorough revision or removal of items with poor parameter estimates, emphasizing continuous monitoring and collaboration between the Ministry of Basic Education and test constructors for ongoing improvement. Additionally, the study recommends stringent review of items with extreme difficulty indices to maintain a balanced assessment.

show abstract

“…In spite of the benefits of multiple-choice tests in educational assessments, it is usually faced with problems of written question often referred to as the stem and problem associated with alternative responses called the distractors, due to non-compliance of the principles of item writing [12]. When items are poorly constructed, particular if the distractors are not plausible enough, it may give room for guessing, thus may lead to poor test quality [8].…”

Section: Introductionmentioning

confidence: 99%

“…When items are poorly constructed, particular if the distractors are not plausible enough, it may give room for guessing, thus may lead to poor test quality [8]. Therefore, in pursuance of excellence in the field of assessment, experts are needed in Nigeria to develop multiple choice test items that meet the expected psychometric properties using new techniques [12].…”

Section: Introductionmentioning

confidence: 99%

The use of measurement frameworks to explore the qualities of test items

Tobih

Ayanwale²,

Ajayi³

et al. 2023

IJERE

View full text Add to dashboard Cite

<span lang="EN-US">Psychometricians and researchers have grown increasingly concern about students’ performance. It is noteworthy from observation in the selected university of education in Nigeria that many of these students do not attend lectures for general courses particularly EDU 222 (tests and measurement) due to overcrowded lecture halls, the classroom climate factor, and other latent factors, such as exam conditions, test anxiety among others. Consequently therefore, students’ performance in EDU 222 does not significantly correlate with these challenges. The characteristics of the item parameter of the Tests and Measurement achievement test at the selected University of Education in Ogun State, Nigeria, were therefore examined in this study using measurement theories. The study employed an ex-post facto research of non-experimental design. Purposive sampling was used to select 6,203 second-year undergraduates offering EDU 222 for the 2018/2019 academic year. To analysis students’ responses to 30 multiple-choice questions on tests and measurement, factor analysis, item calibration, an independent sample t-test with a significance threshold of 0.05 were utilized with empirical reliability of 0.64. The results showed the test items satisfied the three-parameter logistics model and uni-dimensionality assumption. The findings also showed that the item response theory (IRT) eliminated 7 items due to guessing, while the classical test theory (CTT) eliminated 13 items. The test is prone to guessing, thus, helps pupils do better on the exam. To establish parameter estimates during test development, higher education institutions (HEIs) should employ the use of IRT.</span>

show abstract

Calibration of Polytomous Response Mathematics Achievement Test Using Generalized Partial Credit Model

Cited by 4 publications

References 0 publications

The feasibility of computerized adaptive testing of the national benchmark test: A simulation study

The feasibility of computerized adaptive testing of the national benchmark test: A simulation study

Psychometric Assessment of Botswana's 2018 Junior Certificate Agriculture Science Items through Item Response Theory

The use of measurement frameworks to explore the qualities of test items

Contact Info

Product

Resources

About