Modeling Basic Writing Processes From Keystroke Logs

Guo, Hongwen; Deane, Paul; Rijn, Peter W. van; Zhang, Mo; Bennett, Randy Elliot

doi:10.1111/jedm.12172

Cited by 31 publications

(30 citation statements)

References 26 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In terms of the low magnitudes between mixture modeling parameters and the human scores, this is similar to Guo et al’s (2018) finding where they used estimated stable distribution parameters to correlate with human scores. The low magnitudes on the correlations are not surprising, because the goal of the present research is not about replacing human score with the extracted process features.…”

Section: Discussionsupporting

confidence: 75%

“…The advantage of researching the distributions of response time (e.g., pause data) is notable; that is, the parametric methods can provide meaningful summary statistics to represent each student. However, thus far, the only work incorporating a distribution-based analysis in large-scale log data was conducted by Guo et al (2018) . While their work was a promising effort to quantify the writing process, the results were ambiguous, in terms of the meaning of the parameters extracted.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Identifying Mixture Components From Large-Scale Keystroke Log Data

2021

Front. Psychol.

View full text Add to dashboard Cite

In a computer-based writing assessment, massive keystroke log data can provide real-time information on students’ writing behaviors during text production. This research aims to quantify the writing process from a cognitive standpoint. The hope is that the quantification may contribute to establish a writing profile for each student to represent a student’s learning status. Such profiles may contain richer information to influence the ongoing and future writing instruction. Educational Testing Service (ETS) administered the assessment and collected a large sample of student essays. The sample used in this study contains nearly 1,000 essays collected across 24 schools in 18 U.S. states. Using a mixture of lognormal models, the main findings show that the estimated parameters on pause data are meaningful and interpretable with low-to-high cognitive processes. These findings are also consistent across two writing genres. Moreover, the mixture model captures aspects of the writing process not examined otherwise: (1) for some students, the model comparison criterion favored the three-component model, whereas for other students, the criterion favored the four-component model; and (2) students with low human scores have a wide range of values on the mixing proportion parameter, whereas students with higher scores do not possess this pattern.

show abstract

Section: Discussionsupporting

confidence: 75%

Section: Introductionmentioning

confidence: 99%

Identifying Mixture Components From Large-Scale Keystroke Log Data

2021

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…Our own continuing work to advance the state of the art includes investigating the statistical properties of keystroke data (Guo, Deane, van Rijn, Zhang, & Bennett, ), trying to distinguish keyboarding skills from higher level writing skills (Deane et al., ), developing approaches to defining bursts of text personalized to the individual (as opposed to using a uniform threshold for all individuals) (Zhang, Hao, Li, & Deane, ), employing keystroke data to infer whether processes are consistent with the construct the test is designed to measure (Zhang, Zou, et al., ), and trying to model process durations and transitions (e.g., from text generation to editing) to see if they might suggest meaningful demographic group differences (Guo, Zhang, Deane, & Bennett, ). Ultimately, we and others will need to investigate and devise ways to report writing‐process information to teachers and students so that such information contributes to the development of more effective writing and writers.…”

Section: Discussionmentioning

confidence: 99%

Are There Gender Differences inHowStudents Write Their Essays? An Analysis of Writing Processes

Zhang

Bennett

Deane

et al. 2019

Educational Measurement

Self Cite

View full text Add to dashboard Cite

This study compared gender groups on the processes used in writing essays in an online assessment. Middle‐school students from four grades responded to essays in two persuasive subgenres, argumentation and policy recommendation. Writing processes were inferred from four indicators extracted from students’ keystroke logs. In comparison to males, on average females not only obtained higher essay scores but differed from males in their writing processes. Females entered text more fluently, engaged in more macro and local editing, and showed less need to pause at locations associated with planning (e.g., between bursts of text, at sentence boundaries). That these differences were detected after controlling for essay scores suggests that they cannot be attributed solely to disparities in group writing skill.

show abstract

“…Our intention was to develop an estimator that has little costs in regular data and thus can be used on a routine basis. Although our approach was based on the response times in an item, one could of course use different process data for weighting, like, for example, data from keystroke logs (Guo, Deane, van Rijn, Zhang, and Bennett, 2018). There are also some limitations.…”

Section: Discussionmentioning

confidence: 99%

Robust Estimation of Ability and Mental Speed Employing the Hierarchical Model for Responses and Response Times

Ranger

Kuhn

Wolgast

2020

J Educational Measurement

View full text Add to dashboard Cite

Van der Linden's hierarchical model for responses and response times can be used in order to infer the ability and mental speed of test takers from their responses and response times in an educational test. A standard approach for this is maximum likelihood estimation. In real‐world applications, the data of some test takers might be partly irregular, resulting from rapid guessing or item preknowledge. The maximum likelihood estimator is not robust against contamination with irregular data. In this article, we propose a robust estimator of ability and mental speed. The estimator consists of two steps. In the first step, the mental speed is estimated with the estimator of Gervini and Yohai that ignores outlying response times. In the second step, the ability is estimated with an M‐estimator that down weights unusual responses given at unusual response times. This is achieved by combining the hard‐rejection weights of Gervini and Yohai with the M‐estimator suggested by Croux and Haesbroeck for the logistic regression model. The proposed estimator is consistent, almost as efficient as the maximum likelihood estimator in uncontaminated data and robust in contaminated data. The performance of the estimator is analyzed in a simulation study and an empirical example.

show abstract

Modeling Basic Writing Processes From Keystroke Logs

Cited by 31 publications

References 26 publications

Identifying Mixture Components From Large-Scale Keystroke Log Data

Identifying Mixture Components From Large-Scale Keystroke Log Data

Are There Gender Differences inHowStudents Write Their Essays? An Analysis of Writing Processes

Robust Estimation of Ability and Mental Speed Employing the Hierarchical Model for Responses and Response Times

Contact Info

Product

Resources

About