Are You Being Rhetorical? A Description of Rhetorical Move Annotation Tools and Open Corpus of Sample Machine-Annotated Rhetorical Moves

Abstract

Writing analytics has emerged as a sub-field of learning analytics, with applications including the provision of formative feedback to students in developing their writing capacities. Rhetorical markers in writing have become a key feature in this feedback, with a number of tools being developed across research and teaching contexts. However, there is no shared corpus of texts annotated by these tools, nor is it clear how the tool annotations compare. Thus, resources are scarce for comparing tools for both tool development and pedagogic purposes. In this paper, we conduct such a comparison and introduce a sample corpus of texts representative of the particular genres, a subset of which has been annotated using three rhetorical analysis tools (one of which has two versions). This paper aims to provide both a description of the tools and a shared dataset in order to support extensions of existing analyses and tool design in support of writing skill development. We intend the description of these tools, which share a focus on rhetorical structures, alongside the corpus, to be a preliminary step to enable further research, with regard to both tool development and tool interaction

Human‐Centred Learning Analytics: 2019–24

“…We note the current absence of datasets tuned specifically for HCLA . The LA community has published some open datasets (eg, Dietze et al., 2016; Knight et al., 2020; Kuzilek et al., 2017), LearnSphere represents the largest repository of data from online learning, 1 with some LA entries in generic open science repositories 2 . Active initiatives 3 are seeking to transition LA research into open science practices which should encourage more datasets, although safeguards must be in place to protect privacy since educational institutions are understandably wary of sharing student data.…”

A Comparison of Rhetorical Move Analysis by GPT-4 and Humans in Abstracts of Scopus-Indexed Tourism Research Articles

“…In the coding phase, moves or steps are counted as single instances if uninterrupted, but as separate instances if interrupted by different moves or steps . Various assisted annotation software tools for coding rhetorical moves have been examined, such as Mover, Research Writing Tutor, Academic Writing Analytics (AWA), and AcaWriter tools (Knight et al, 2020). However, to the researchers' knowledge, there has been no study to date that utilizes ChatGPT (GPT-4) for the coding of rhetorical moves within academic texts.…”