Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays

Kumar, Rahul; Mathias, Sandeep; Saha, Sriparna; Bhattacharyya, Pushpak

doi:10.18653/v1/2022.naacl-main.106

Cited by 6 publications

(6 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 3 presents the outcomes from a combination of 10 cross-validation experiments and one additional, thereby facilitating a holistic assessment of model capabilities. Notably, Trans-BERT-MS-ML-R [23] and MTL-CNN-BiLSTM [16] were performed solely within the scope of single-experiment score predictions.…”

Section: Performance Comparisonmentioning

confidence: 99%

“…In Table 5, the experiments for BERT were implemented by the authors. Specific numerical results for MTL-CNN-BiLSTM are absent due to the format of the data presentation in [16]. For a comparative visual comparison based on those bar graphs, refer to Figure 2.…”

Section: Performance Comparisonmentioning

confidence: 99%

“…Some studies [34–36] have highlighted the importance of trait utilization, emphasizing the adoption of multi‐task learning (MTL) approaches for multi‐trait evaluation. In this context, [16] presented a thorough comparison between single‐task learning (STL) and MTL, demonstrating the advantages of the latter. Numerous studies have investigated inherent trait characteristics to provide a deeper understanding and more accurate evaluation of essay quality.…”

Section: Related Workmentioning

confidence: 99%

“…Trans-BERT-MS-ML-R [26]: Leveraging BERT for essay scoring, this model uses a joint learning method to generate multiscale essay representations and incorporates multiple loss functions alongside transfer learning techniques. MTL-CNN-BiLSTM [16]: This model combines CNN-BiLSTM using an MTL approach with holistic scoring as the primary task and multi-trait scoring as an auxiliary task. BERT [12]: The BERT model pretrains deep bidirectional representations from text and achieves state-of-the-art performance in numerous natural language processing tasks.…”

Section: Multi-task Learningmentioning

confidence: 99%

“…Evaluating trait‐specific attributes can provide detailed feedback and enrich the understanding and transparency of an essay's overall quality. Previous work [16] considered essay traits an auxiliary task, mainly concentrating on using their scores to improve holistic score prediction. However, this approach handles traits secondary to holistic accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Dual‐scale BERT using multi‐trait representations for holistic and trait‐specific essay grading

Cho,

Huang,

Kwon

2024

ETRI Journal

View full text Add to dashboard Cite

As automated essay scoring (AES) has progressed from handcrafted techniques to deep learning, holistic scoring capabilities have merged. However, specific trait assessment remains a challenge because of the limited depth of earlier methods in modeling dual assessments for holistic and multi‐trait tasks. To overcome this challenge, we explore providing comprehensive feedback while modeling the interconnections between holistic and trait representations. We introduce the DualBERT‐Trans‐CNN model, which combines transformer‐based representations with a novel dual‐scale bidirectional encoder representations from transformers (BERT) encoding approach at the document‐level. By explicitly leveraging multi‐trait representations in a multi‐task learning (MTL) framework, our DualBERT‐Trans‐CNN emphasizes the interrelation between holistic and trait‐based score predictions, aiming for improved accuracy. For validation, we conducted extensive tests on the ASAP++ and TOEFL11 datasets. Against models of the same MTL setting, ours showed a 2.0% increase in its holistic score. Additionally, compared with single‐task learning (STL) models, ours demonstrated a 3.6% enhancement in average multi‐trait performance on the ASAP++ dataset.

show abstract

Section: Performance Comparisonmentioning

confidence: 99%

Section: Performance Comparisonmentioning

confidence: 99%