Reporting Writing Process Feedback in the Classroom. Using Keystroke Logging Data to Reflect on Writing Processes

Vandermeulen, Nina; Leijten, Mariëlle; Waes, Luuk Van

doi:10.17239/jowr-2020.12.01.05

Cited by 42 publications

(36 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These and many other new trends in students' writing practices call for critical reflections on longstanding understandings of writing and serious research efforts to examine and assess such emerging trends. Recent studies such as Vandermeulen et al (2020) shed inspiring light on the value of reaping educational benefits from quantified and visualized reports. Since pausing can reveal much about the complex cognitive underpinnings of writing and the writer's cognitive styles and metacognitive activities, students' pause profiles can be fruitfully used to inform the writing classroom and to make it more effective and engaging for learners at different levels.…”

Section: Limitations and Implicationsmentioning

confidence: 99%

Profiling the pausing behaviour of EFL learners in real-time computer-aided writing: a multi-method case study

Shen

Chen

2021

Asian. J. Second. Foreign. Lang. Educ.

View full text Add to dashboard Cite

Technologically enhanced means and devices in language education and research have enabled an in-depth exploration of the dynamics of writing. This study investigated the pausing behaviour of eight Chinese English as a Foreign Language (EFL) learners at the tertiary level in performing an online writing task. Quantitative and qualitative data were collected using a combination of methods and techniques, including keystroke logging, screen recording videos, think-aloud protocols, and stimulated recall interviews to establish a profile of each learner’s pausing behaviour. The learners’ pause profiles were extensively analyzed with a comparative focus on similarities and differences in EFL learners’ pausing behaviour across writing skill levels. Overall, the findings revealed a general tendency for the learners to pause most frequently at a low text unit level, i.e., the lexical level. More specifically, less-skilled writers tended to pause more frequently than more-skilled writers at lower-level text units, whilst more-skilled writers chose to make more strategic pauses to gain overall control of their writing. Furthermore, these findings help reveal the intricate self-monitoring patterns that undergird individual writer’s pausing behaviour and relate these patterns to self-monitoring awareness, writing knowledge and experience, and writing habit. This small-scale multi-method study offers a glimpse into how EFL learners at different skill levels would respond to a real-time online writing task by using resources at their disposal and under conscious monitoring. Methodologically, it adds empirical evidence to previous literature on researching the computer-aided writing process with computer-aided tools and considers productive complementation and triangulation across research approaches and paradigms.

show abstract

Section: Limitations and Implicationsmentioning

confidence: 99%

Profiling the pausing behaviour of EFL learners in real-time computer-aided writing: a multi-method case study

Shen

Chen

2021

Asian. J. Second. Foreign. Lang. Educ.

View full text Add to dashboard Cite

show abstract

“…Only some tools provide additional resources to aid the writing process. For example, Criterion provides a portfolio history of drafts, to gain insight into one's writing progress over time (Link et al, 2014); eWritingPal includes lecture videos with animated agents to teach strategies for pre-writing, drafting, and revising (Roscoe et al, 2014); ThesisWriter suggests some strategies for improving research report writing (Rapp & Kauf, 2018), and the Inputlog Process Report consists of an automatically generated text file addressing different perspectives of the writing process: pausing, revision, source use, and fluency (Vandermeulen, Leijten, & Van Waes, 2020). Apart from the latter, none of these tools yet collect evidence from the writing process nor provide feedback on specific writing processes.…”

Section: Data About Writing Processesmentioning

confidence: 99%

How to provide automated feedback on the writing process? A participatory approach to design writing analytics tools

Conijn

Martínez-Maldonado

Knight

et al. 2020

Computer Assisted Language Learning

View full text Add to dashboard Cite

“…It has been quite difficult to create a pedagogical context in which students were challenged to reflect on their writing process. However, given the current developments in keystroke analysis, we can automatically present students with both detailed and wide perspectives on different aspects of their writing process, e.g., based on the features extracted in the current study (related to their pausing or revision behavior, fluency, or source usage-for an example of a process feedback report, see Vandermeulen et al, 2020). Although these might not provide 100% accurate predictions of students' writing quality, they might still be used by students to reflect on their process.…”

Section: Implications For Automated Writing Process Analysis and Future Researchmentioning

confidence: 99%

“…This illustrates that there might not be a single 'best' writing process. A small case study showed that a feedback report including the process graph helped students to reflect on their writing process and even enabled them to identify ways to improve their writing process (Vandermeulen et al, 2020).…”

Section: Implications For Automated Writing Process Analysis and Future Researchmentioning

confidence: 99%

“…This approach has the additional advantage that it may be used to provide feedback both during and on the writing process. Feedback on the writing process is more powerful for deep processing than feedback on the product, as it can not only be used to improve the current task, but also to enhance (process) skills that can be transferred to future tasks (Hattie & Timperley, 2007;Vandermeulen et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Early prediction of writing quality using keystroke logging

Conijn

Cook

Zaanen³

et al. 2021

Int J Artif Intell Educ

View full text Add to dashboard Cite

Feedback is important to improve writing quality; however, to provide timely and personalized feedback is a time-intensive task. Currently, most literature focuses on providing (human or machine) support on product characteristics, especially after a draft is submitted. However, this does not assist students who struggle during the writing process. Therefore, in this study, we investigate the use of keystroke analysis to predict writing quality throughout the writing process. Keystroke data were analyzed from 126 English as a second language learners performing a timed academic summarization task. Writing quality was measured using participants’ final grade. Based on previous literature, 54 keystroke features were extracted. Correlational analyses were conducted to identify the relationship between keystroke features and writing quality. Next, machine learning models (regression and classification) were used to predict final grade and classify students who might need support at several points during the writing process. The results show that, in contrast to previous work, the relationship between writing quality and keystroke data was rather limited. None of the regression models outperformed the baseline, and the classification models were only slightly better than the majority class baseline (highest AUC = 0.57). In addition, the relationship between keystroke features and writing quality changed throughout the course of the writing process. To conclude, the relationship between keystroke data and writing quality might be less clear than previously posited.

show abstract

Reporting Writing Process Feedback in the Classroom. Using Keystroke Logging Data to Reflect on Writing Processes

Cited by 42 publications

References 15 publications

Profiling the pausing behaviour of EFL learners in real-time computer-aided writing: a multi-method case study

Profiling the pausing behaviour of EFL learners in real-time computer-aided writing: a multi-method case study

How to provide automated feedback on the writing process? A participatory approach to design writing analytics tools

Early prediction of writing quality using keystroke logging

Contact Info

Product

Resources

About