Supporting Spanish Writers using Automated Feedback

Abstract

We present a tool that provides automated feedback to students studying Spanish writing. The feedback is given for four categories: topic development, coherence, writing conventions, and essay organization. The tool is made freely available via a Google Docs add-on. A small user study with post-secondary level students in Mexico shows that students found the tool generally helpful and that most of them plan to continue using it as they work to improve their writing skills. In an analysis of 6 months of user data, we see that a small number of users continue to engage with the app, even outside of planned user studies.

A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring

“…Automated writing support (AWS) is one of the most actively researched areas related to educational tools: With respect to college-level students, present Writing Mentor, a writing evaluation tool for scientific writing in English, which evaluates students' texts along multiple criteria, including, among others, coherence, topic development, scientific conventions as well as orthographic and grammatical correctness. 2 An extension of Writing Mentor to Spanish has recently been released (Cahill et al, 2021). Also working on Spanish, the system by González-López et al (2020) specifically evaluates the methodology section of Mexican college students' theses in engineering subjects.…”

A Design Space for Intelligent and Interactive Writing Assistants

“…Advancements in AI accelerators and the availability of large amounts of data have led to an evolution in model architectures, 5 which we capture as the following four types. First, rule-based models rely on pre-defined logic, lookup tables, regular expressions, or other similar heuristic approaches that are deterministic in nature [10,29,90,221]. For statistical machine learning (ML) models, we consider models that are trained from scratch on historical data, are not necessarily "deep" (as in deep neural networks), and are used to make future predictions (e.g., support vector machines and logistic regression) [112,122,195,271].…”

“…However, providing the natural language outputs of such models requires the designer to give up a great deal of control over the possible outputs. In sensitive contexts such as topics like mental health [191,195,255] or to leverage the benefit of high-quality source material, designers may choose to trade off the flexibility of an NLP model for the control of a premade, curated list of responses [232], from which an output is selected by the system [29,37,220]. To increase personalization, these pre-curated responses may also be dynamically customized [191].…”

“…First, systems designed as agents employ designs meant to evoke humanlike interaction, including roles like "collaborator" or "co-writer" [36,148], "dialog partner" [191,249], "assistant" [80], and "companion" [84]. Techniques for evoking the agent metaphor include character-by-character text rendering to simulate typing [121], avatars [29], implicit and explicit conversational interaction [191], and first-person conversational styles [195]. In contrast, other systems present as tools, where there is no sense of interacting with another agent.…”

“…Systems can access different types of user data to produce the desired outputs. Typically, the user's input text is the primary source of data; the current writing progress can be used to generate completions [7,148] or provide feedback [29,77]. It is also important to allow users to have additional controls over the text generations, which we characterize as additional data.…”