Hierarchical Multi-task Learning for Organization Evaluation of Argumentative Student Essays

Abstract

<jats:p>Organization evaluation is an important dimension of automated essay scoring. This paper focuses on discourse element (i.e., functions of sentences and paragraphs) based organization evaluation. Existing approaches mostly separate discourse element identification and organization evaluation. In contrast, we propose a neural hierarchical multi-task learning approach for jointly optimizing sentence and paragraph level discourse element identification and organization evaluation. We represent the organization as a grid to simulate the visual layout of an essay and integrate discourse elements at multiple linguistic levels. Experimental results show that the multi-task learning based organization evaluation can achieve significant improvements compared with existing work and pipeline baselines. Multiple level discourse element identification also benefits from multi-task learning through mutual enhancement.</jats:p>

Multi-Task Learning in Natural Language Processing: An Overview

“…In addition, deep cascade MTL [33] adds both residual connections and cascade connections to a single-trunk parallel MTL model with supervision at different levels, where residual connections forward hidden representations and cascade connections forward output distributions of a task to the prediction layer of another task. [112] includes the output of the low-level discourse element identification task in the organization grid, which consists of sentence-level, phrase-level, and document-level features of an essay, for the primary essay organization evaluation task. In [107], the word predominant sense prediction task and the text categorization task share a transformer-based embedding layer and embeddings of certain words in the text categorization task could be replaced by prediction results of the predominant sense prediction task.…”

“…where 𝜆 increases linearly from 0 to 1 in the training process. In [112], three tasks are jointly optimized, including the primary essay organization evaluation (OE) task and the auxiliary sentence function identification (SFI) and paragraph function identification (PFI) tasks. The two lower-level auxiliary tasks are assumed to be equally important with weights set to 1 (i.e., 𝜆 𝑆𝐹 𝐼 = 𝜆 𝑃 𝐹 𝐼 = 1) and the weight of the OE task is set as…”

IFlyEA: A Chinese Essay Assessment System with Automated Rating, Review Generation, and Recommendation

“…A discourse element is represented by combining its distributed semantic vector and a manually constructed feature vector (Song et al, 2015). The learning framework is based on hierarchical multi-task learning (Song et al, 2020b), which jointly optimizes sentence and paragraph level discourse element identification and organization evaluation. Evaluation shows that some minority discourse elements, such as thesis and ideas, are more difficult to recognize, and organization evaluation of argumentative essays is still challenging due to the lack of large-scale training data.…”

Argument Pair Extraction with Mutual Guidance and Inter-sentence Relation Graph

“…Most existing studies in the field of argumentaion mining focus on monological argumentation, such as argumentation structure parsing (Stab and Gurevych, 2017;Afantenos et al, 2018;Kuribayashi et al, 2019;Hua et al, 2019b;Morio et al, 2020), automated essay scoring Ke et al, 2018;Song et al, 2020), argument quality assessment Gretz et al, 2020;Lauscher et al, 2020), argumentation strategies modeling (Khatib et al, , 2017, etc.…”