The Effects of Administration and Response Modes on Grade 1–2 Students’ Writing Performance

Abstract

This study aimed to investigate how Grade 1-2 English language learners (ELLs) differ in their performance on a writing test in two test modes: paper and online. Participants were 139 ELLs in the United States. They completed three writing tasks, representing three test modes: (1) a paper in which students completed their writing using a paper-and-pencil format, (2) online-handwriting in which students viewed the prompt online and handwrote responses, and (3) online-keyboarding in which learners viewed the prompt online and keyboarded responses. Up to 20 students were tested during each 90-minute session, during which researchers observed two or three students to examine any differences in their writing behaviors across tasks. Immediately after students completed the tasks, researchers asked them about their preferred test modes. Students' writings were analyzed quantitatively and qualitatively in terms of task completion, fluency, and complexity. Interview data were analyzed qualitatively. Findings suggest the handwriting task is the optimal mode for students in Grades 1-2. Yet the majority of ELLs observed said they preferred the keyboarding task over the handwriting tasks. Study results provide practical implications for writing test administration for K-12 ELLs.

Validity of score interpretations on an online English placement writing test

“…The present study findings suggest that, in general, EPT test-takers tended to hold self-confidence in CBW tests regardless of L2 writing ability for reasons related to prior CBW experiences (convenience in writing, familiarity with CBW, and keyboard typing). The finding lent support for results from previous research showing that the majority of test-takers held similarly positive attitudes towards CBW tests (e.g., Kim et al, 2018 ; Kim et al, 2019 ; Lee, 2004 ), due to useful word processing-related functions reasons, like expedited writing (e.g., insertion, deletion, and movement capacities), and the professional and neat appearance of typed texts (Brunfaut et al, 2018 ). The present and previous findings together indicate widespread favorable attitudes toward (self-confidence in) CBW tests among various test-taker populations (including EPT test-takers) and test-takers’ overall familiarity with CBW that involves typing and use of word processing tools.…”

“…The increasing computer usage indicates CBW tests provide “realistic contexts for the production of student work by having the tasks and processes, as well as time and resources, parallel those in the real world” (Messick, 1994 , p. 18). In addition, CBW tests are convenient, in that many test-takers can take the tests without the constraints of time and location (e.g., Kim et al, 2019 ). Computer testing also saves costs associated with printing and shipping paper (Way et al, 2008 ), although advancements in computerization and required security measures come at a high cost as well.…”

“…Test-takers’ perceptions of and preferences for computerized testing are commonly measured aspects of computer familiarity that have drawn much attention in cross-modality studies. According to the literature, an increasing number of test-taker populations at a whole-group level positively perceive or prefer CBW assessment for a range of reasons, including convenience and efficiency of the test and confidence in and adeptness at typing (e.g., Feng et al, 2019 ; Kim et al, 2019 ; Lee, 2004 ; Yu & Iwashita, 2021 ).…”

“…many test-takers can take the tests without the constraints of time and location (e.g., Kim et al, 2019). Computer testing also saves costs associated with printing and shipping paper (Way et al, 2008), although advancements in computerization and required security measures come at a high cost as well.…”

“…Some international testing companies (e.g., TOEFL iBT Home Edition) and many higher education institutions' tests (e.g., Iowa State University's English writing placement test) in developing and developed countries have successfully shifted assessments to online, computer-based formats in recent years (Babbar & Gupta, 2021;Isbell & Kremmel, 2020). A majority of test-takers consistently perceive this transition as positive regardless of the test-takers' age (e.g., schoolchildren, adults) (e.g., Kim et al, 2019). The favorable attitudes seem to champion CBW testing, though further supporting evidence is needed to better justify the shift to online testing platforms.…”