Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0

Toni, Francesco De; Akiki, Christopher; Rosa, Javier de la; Fourrier, Clémentine; Manjavacas, Enrique; Schweter, Stefan; Strien, Daniel van

doi:10.18653/v1/2022.bigscience-1.7

Cited by 1 publication

(1 citation statement)

References 18 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While there has been some work regarding zero-shot entity recognition in historic German newspaper [33], to the best of our knowledge, we are the first to apply zero-shot aspectbased sentiment classification to German texts. We present an in-depth comparison between the zero-shot approach and specifically trained models, fine-tuned on hand-coded data, for the application on historic German texts.…”

Section: Related Workmentioning

confidence: 99%

An Evaluation of a Zero-Shot Approach to Aspect-Based Sentiment Classification in Historic German Stock Market Reports

Borst,

Wehrheim,

Niekler

et al. 2023

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

One critical aspect that remains in the application of state-of-the-art neural networks to text analysis in applied research is the continued requirement for manual data annotation. In computer science research, there is a strong focus on maximizing the data efficiency of fine-tuning language models. This has led to the development of zero-shot text classification methods, which promise to work effectively without requiring fine-tuning for the specific task at hand. In this paper, we conduct an in-depth analysis of aspect-based sentiment analysis in historic German stock market reports to evaluate the reliability of this promise. We present a comparison of a zero-shot approach with a meticulously fine-tuned three-step process of training and applying text classification models. This study aims to empirically assess the reliability of zero-shot text classification and provide justification for the potential benefits it offers in terms of reducing the burden of data labeling and training for analysis purposes. The findings of our study demonstrate a strong correlation between the sentiment time series generated through aspect-based sentiment analysis using the zero-shot approach and those derived from the fine-tuned supervised pipeline, validating the viability of the zero-shot approach. While the zero-shot pipeline exhibits a tendency to underestimate negative examples, the overall trend remains discernible. Additionally, a qualitative analysis of the linguistic patterns reveals no explicit error patterns. Nevertheless, we acknowledge and discuss the practical and epistemological obstacles associated with employing zero-shot algorithms in untested domains.

show abstract

Section: Related Workmentioning

confidence: 99%