Combined Deployable Keystroke Logging and Eyetracking for Investigating L2 Writing Fluency

Abstract

Although fluency is an important subconstruct of language proficiency, it has not received as much attention in L2 writing research as complexity and accuracy have, in part due to the lack of methodological approaches for the analysis of large datasets of writing-process data. This article presents a method of time-aligned keystroke logging and eye-tracking and reports an empirical study investigating L2 writing fluency through this method. Twenty-four undergraduate students at a private university in Turkey performed two writing tasks delivered through a web text editor with embedded keystroke logging and eye-tracking capabilities. Linear mixed-effects models were fit to predict indices of pausing and reading behaviors based on language status (L1 vs. L2) and linguistic context factors. Findings revealed differences between pausing and eye-fixation behavior in L1 and L2 writing processes. The article concludes by discussing the affordances of the proposed method from the theoretical and practical standpoints.

Exploring Second Language Writers’ Pausing and Revision Behaviors

“…The same pattern was observed for the two task types and for the two groups of writers (L1 vs. L2), although the L2 writers, as expected, generally paused longer at each textual location. Spelman Miller's findings have been confirmed in a number of more recent studies employing keystroke logging (Chukharev-Hudilainen et al, 2019;Révész, Kourtali et al, 2017;Révész, Michel et al, 2017;Van Waes & Leijten, 2015). Among these, Van Waes and Leijten's work is of particular significance because the researchers used four different pause thresholds (200, 500, 1000, 2000ms) when studying L2 fluency behaviors.…”

“…Although there is a growing number of studies utilizing a combination of methods to tap the writing process (e.g., Gánem-Gutiérrez & Gilmore, 2018;Khuder & Harwood, 2015;Révész, Kourtali et al, 2017;Stevenson et al, 2006), only few such L2 studies (Chukharev-Hudilainen et al, 2019;Révész, Kourtali et al, 2017) have looked into pausing behaviors according to textual location. Révész, Kourtali et al (2017) studied the writing behaviors of 73 advanced L2 writers carrying out tasks of differential cognitive complexity.…”

“…Although Révész, Kourtali et al (2017) and Chukharev-Hudilainen et al (2019) provide more detailed accounts of the processes underlying pausing behaviors than studies that have used keystroke logging alone, they are not without shortcomings. Révész, Kourtali et al (2017) sheds little light on participants' viewing behaviors during pauses, whereas Chukharev-Hudilainen et al (2019) provides no direct information about participants' thought processes while writing. To address these limitations, the present study made use of all three data sources-keystroke logging, eye-tracking, and verbal protocol-to better uncover the cognitive processes associated with pausing at various textual units.…”

“…Besides theoretical and practical considerations, the enhanced research effort at studying pausing and revision behaviors is probably due to recent technological developments, which allow for obtaining a more fine-grained description of observable pausing and revision phenomena and, hence, for making more valid inferences about corresponding cognitive processes. For many years, verbal protocols were the preferred method in writing process research (e.g., Roca de Larios et al, 2008), but, increasingly, L2 researchers also utilize more novel tools such as keystroke logging (Spelman Miller, 2000;Stevenson, Schoonen, & de Glopper, 2006) and eye-tracking to examine pausing and revision behaviors (Chukharev-Hudilainen, Feng, Saricaoglu, & Torrance, 2019;Gánem-Gutiérrez & Gilmore, 2018;Révész, Michel, & Lee, 2017). A few studies have additionally succeeded in combining multiple techniques to gain a more complete picture of pausing and revision phenomena and underlying cognitive processes (e.g., Chukharev-Hudilainen et al, 2019;Khuder & Harwood, 2015;Révész, Kourtali, & Mazgutova, 2017;Stevenson et al, 2006).…”

A product- and process-oriented tagset for revisions in writing

“…Our dataset was created by sampling writing-process data from a large data store containing anonymized (i.e., not containing any personally identifiable information or links to research participants) writing-process log files. These log files were recorded as part of various prior research studies conducted using CyWrite, a web-based word-processing tool that collects keystroke data and eye fixations during writing (Chukharev-Hudilainen, 2019;Chukharev-Hudilainen, Saricaoglu, Torrance, & Feng, 2019;Ranalli, Feng, & Chukharev-Hudilainen, 2018). In addition, the CyWrite tool offers playback functionality to replay the screen recording combined with the gaze-point marker.…”