Item Difficulty as a Source of Variability in Student Achievement in the West African Secondary School Certificate Examination (WASSCE): Application of Generalizability Theory

Abstract

Students’ achievement in core subjects in the West African Secondary School Certificate Examination (WASSCE) has been a subject of concern for stakeholders due to poor performance. The purpose of this study is to assess whether item difficulty is a significant source of variability to the measurement of students’ achievement in WASSCE in Ghana, using the generalizability theory. The study had three specific objectives: (1) to examine the sources of variability (e.g., item) to students’ achievement in WASSCE in Ghana, (2) to assess the dependability of students’ responses in relation to their ability, (3) evaluate how many items are sufficient to provide an optimum measure of student achievement. The one-facet crossed random design was adopted as the research design. Data were obtained from students’ achievement in 2015 WASSCE in the four subjects, namely, Mathematics, Science, Social Studies and English Language. The generalized analysis of variance (GENOVA) was used for the analysis, conducting both G-study and D-study. The analysis revealed that the findings from this study showed that item difficulty had minimal effect on the variability in students’ achievement in English Language and Mathematics. The result further revealed that item difficulty had a significant effect on variability in students’ achievement in Science and Social Studies. Only the English Language test showed relatively low reliability. The study recommended the item difficulty and content structure of the Science and Social Studies multiple-choice test should be evaluated by WAEC officials and their examiners.

Psychometric Assessment of Botswana's 2018 Junior Certificate Agriculture Science Items through Item Response Theory

“…It sounds good that the items had varying difficulty levels and thus, items that were dysfunctional were few. This understanding centers on the fact that item difficulty may contribute significantly to the variations in students' performance (Cobbinah et al, 2022). The findings appear to contradict observations of previous studies that examined the difficult indicators of national-wide exams revealed a moderate level of difficulty (Deborah et al, 2020;Ibrahim, 2023;Onah & Jiwueze, 2015).…”

Modeling unit non-response and validity of online teaching evaluation in higher education using generalizability theory approach

“…Furthermore, GT removes the traditional variations between validity and reliability (Allal and Cardinet, 1997). Due to the advantages of the GT over the CMT, it has been applied in several studies to address research problems in different areas of study (García-García et al, 2013;Morales-Sánchez et al, 2020;Reigal et al, 2020;Cobbinah et al, 2022). Shavelson and Webb (1991) argued that GT is a protracted form of the CMT for four reasons: 1) The GT has the ability to estimate several sources of variability in a single computation, 2) the use of GT guarantees the estimation of the magnitude of each source of variability, 3) GT permits the computation of two different errors of measurement and reliability coefficients and thus, makes it possible to take relative and absolute decisions, and 4) GT allows for realistic measurement decisions to be made to reduce measurement errors to the barest minimum based on specific purposes.…”

“…Furthermore, GT removes the traditional variations between validity and reliability ( Allal and Cardinet, 1997 ). Due to the advantages of the GT over the CMT, it has been applied in several studies to address research problems in different areas of study ( García-García et al, 2013 ; Morales-Sánchez et al, 2020 ; Reigal et al, 2020 ; Cobbinah et al, 2022 ).…”