The Sensitivity of a Scenario-Based Assessment of Written Argumentation to School Differences in Curriculum and Instruction

Leveraging process data to assess adults’ problem-solving skills: Using sequence mining to identify behavioral patterns across digital tasks

“…Process data have been used extensively in the educational literature in the context of educational data mining, learning analytics and artificial intelligence. For example, process data on patterns of keystrokes has been used to examine writing behavior (e.g., Deane et al, 2020;Guo et al, 2020). Process data has also been used to examine if learning can be fostered through the use of AI-operated instructional environments (e.g., Biswas et al, 2005;; to monitor the development of problem-solving skills in game-based assessments (e.g., Shute et al, 2016), and how AI systems can be used to measure human intelligence (e.g., Embreston, 2014).…”

The Platformization of Writing Instruction: Considering Educational Equity in New Learning Ecologies

“…Researchers noted that because “written text is crucial for professional and academic success” (Snow et al, 2015, p. 40), students who “cannot write well are disadvantaged in salary positions and often find coursework too difficult to complete” (Wang et al, 2020). These imagined writing purposes were reflected in the most common written forms featured in the studies, including essay responses to standardized prompts (Allen et al, 2019; Deane et al, 2021), class essays (e.g., Huang & Wilson, 2021; Lim & Phua, 2019; McCarthy et al, 2022), summaries of class readings (e.g., Sung et al, 2016), and explanations of concepts (e.g., Gerard & Linn, 2022; H. S. Lee et al, 2021).…”

Predictors of middle school students’ perceptions of automated writing evaluation

“…Differences in perceptions may also or instead be due to district-specific differences in implementation (see Table 2 ). Indeed, prior research indicates that AWE implementation varies across districts ( Deane et al, 2021 ; Mayfield & Butler, 2018 ). Moreover, AWE implementation occurs within a curricular context, and teachers may perceive some curricula as being less amenable to AWE integration (see Link et al, 2014 ) or teachers may selectively utilize certain AWE functions, such as its writing practice versus peer review functionality, as occurred in the present study and in prior research ( Wilson, Huang et al, 2021 ).…”

“…Based on students' open-ended survey responses, one possibility might be that more proficient writers did not receive the feedback they needed from MI Write, perhaps due to noted limitations in AWE feedback on higher-level writing skills ( Deane, 2013 ). Researchers and AWE developers should continue to expand the assessment and feedback capabilities of their systems to provide feedback on higher-level writing skills (e.g., Correnti et al, 2022 ; Raković et al, 2021 ), as well as aspects of the writer's process, not just the writing product (see Deane et al, 2021 ; Raković et al, 2022 ).…”

“…In our investigation, we considered various factors that could potentially influence students' perceptions of AWE. First, we examined variables that accounted for the district and classroom context in which MI Write was implemented, as prior research has shown that AWE implementation can vary across districts and classrooms ( Deane et al, 2021 ; Mayfield and Butler, 2018 , Wilson, Myers, & Potter, 2022 ) and such variation is reasonably related to students’ perceptions of AWE.…”