Detecting Randomness Effect among Raters in Physics Essay Items using Many-Facet Rasch Measurement

Abstract

This study sought to detect randomness effects among raters in physics essay items using Many-Facet Rasch Measurement. The research design adopted for this study is descriptive research design based on survey method. The population of the study comprised eighty-eight (88) public schools in all the local government areas with a physics student population of 3,642 students and ninety-four (94) physics teachers in all the Senior Secondary Schools in Uyo Senatorial District for the 2022/2023 academic session. Three hundred and sixty-four (364) SSS3 physics students and 37 physics teachers from the 31 selected secondary schools in Uyo Senatorial District were sampled using multistage sampling technique for effective selection. The multi-stage sampling technique was adopted for the study. The instrument used for data collection was Physics Achievement Test (PAT) obtained from WAEC and NECO 2020 Physics Essay items with reliability coefficients of 0.91 and 0.90 respectively. The finding revealed that most of the individual raters commit randomness effect when rating physics essay Items. It also revealed there is a significant difference at the rater's group level exhibiting randomness effect when rating physics essay items, which implies that there is no group-level randomness effect present among raters when rating physics essay items. We concluded that rater effects are sources of variance in performance ratings that are associated with the raters' behaviour and not the actual performance of the ratee. It was recommended that raters should follow the rating guidelines to reduce the impact of randomness in their ratings and provide more accurate and objective evaluations.