Development and Evaluation of Scoring Support System in Writing Test for Subject Teachers and Classroom Teachers

Abstract

The purpose of this study is to develop and evaluate a system to support the scoring of writing tests in primary and secondary schools employing the subject-teacher and the classroom-teacher systems. To this end, we employ the previously proposed image-skewered method, in addition to an area setting package function, to address commercially available tests used by classroom teachers and a multiple class selection function to score tests completed by students from multiple classes taught by the same subject teacher. Further, a function is added to operate only in the case of incorrect answers to reduce the number of operations required to assess the correctness of answers. To evaluate the system, the time taken to manually score actual test papers was compared with the time taken to do so using the proposed system. The automatic scoring method exhibited a time reduction by 44.0% for subject teachers and 42.9% for classroom teachers. These results indicate that the system functions effectively as a scoring support system to reduce the time consumed for scoring tasks for teachers working in primary and secondary schools.

Characteristics and Cognitive Process of Teachers’ Test Scoring Work in Japanese Elementary and Secondary Education

“…Nakagawa et al [17] observed and surveyed test scoring in the manual processes (i.e., penand-paper), especially in correct/incorrect judgments, and classified the method of correct/incorrect judgments as follows:…”

“…Nakagawa and Horita [27] conducted semi-structured interviews on junior high school teachers in line with the details of the scoring methods as defined by Nakagawa et al [17], to survey what they attach importance to during each task process of the scoring. Typification was performed on the results of each interview, with regard to the cognitive processes of teachers in each task process of the scoring, and the question items in the questionnaire survey were created.…”

“…Nakagawa et al [17] surveyed the characteristics (i.e., number of questions, number of tests in a year, and test takers per test) of the tests used for evaluation in elementary, junior high schools and senior high schools, based on prior studies [14][22][23] [24]. They are compiled in Table 1.…”

“…• the number of tests in a year is less than that of elementary schools, and • the number of questions and test takers per test are relatively large. Nakagawa et al [17] arranged the flow of test scoring by dividing them into three categories by referencing prior studies [25][26]. The first step is "preparation" (i.e., producing and duplicating test questions).…”

“…Elementary school teachers use "unit test" scores and junior high school teachers use "periodical test" scores for creating the students' report cards [14]. Several scoring support systems have been developed in Japan to reduce teachers' excessive workload [15][16] [17]. MEXT has also published a case study demonstrating how a scoring support system can decrease test scoring time [18].…”