Acceptability Judgements via Examining the Topology of Attention Maps

Cherniavskii, Daniil; Tulchinskii, Eduard; Mikhailov, Vladislav; Proskurina, Irina; Kushnareva, Laida; Artemova, Ekaterina; Barannikov, Serguei; Piontkovskaya, Irina; Piontkovski, Dmitri; Burnaev, Evgeny

doi:10.48550/arxiv.2205.09630

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first issue is the subjective nature of acceptability judgment, which can vary according to context and language and is influenced by syntactic or semantic features as well as pragmatic and dialogic ones. Therefore, models facing this task must be able to identify and capture complex linguistic structures [32] and exploit cross-lingual approaches to generalize between languages [33].…”

Section: Related Workmentioning

confidence: 99%

Raising the Bar on Acceptability Judgments Classification: An Experiment on ItaCoLA Using ELECTRA

Guarasci,

Minutolo,

Buonaiuto

et al. 2024

Electronics

View full text Add to dashboard Cite

The task of automatically evaluating acceptability judgments has relished increasing success in Natural Language Processing, starting from including the Corpus of Linguistic Acceptability (CoLa) in the GLUE benchmark dataset. CoLa spawned a thread that led to the development of several similar datasets in different languages, broadening the investigation possibilities to many languages other than English. In this study, leveraging the Italian Corpus of Linguistic Acceptability (ItaCoLA), comprising nearly 10,000 sentences with acceptability judgments, we propose a new methodology that utilizes the neural language model ELECTRA. This approach exceeds the scores obtained from current baselines and demonstrates that it can overcome language-specific limitations in dealing with specific phenomena.

show abstract

Section: Related Workmentioning

confidence: 99%

Raising the Bar on Acceptability Judgments Classification: An Experiment on ItaCoLA Using ELECTRA

Guarasci,

Minutolo,

Buonaiuto

et al. 2024

Electronics

View full text Add to dashboard Cite

show abstract

“…[18][19][20][21] TDA has been used in NLP for movie genre detection, 22 textual entailment, 23 document summarization, 24 and analysis of sentence embeddings. 25 The topology of the attention layers has been leveraged for text classification, 26,27 acceptability judgments, 28 and robustness against adversarial attacks. 29…”

Section: Related Workmentioning

confidence: 99%

TopoBERT: Exploring the topology of fine-tuned word representations

Rathore

Zhou

Srikumar

et al. 2023

Information Visualization

View full text Add to dashboard Cite

Transformer-based language models such as BERT and its variants have found widespread use in natural language processing (NLP). A common way of using these models is to fine-tune them to improve their performance on a specific task. However, it is currently unclear how the fine-tuning process affects the underlying structure of the word embeddings from these models. We present TopoBERT, a visual analytics system for interactively exploring the fine-tuning process of various transformer-based models – across multiple fine-tuning batch updates, subsequent layers of the model, and different NLP tasks – from a topological perspective. The system uses the mapper algorithm from topological data analysis (TDA) to generate a graph that approximates the shape of a model’s embedding space for an input dataset. TopoBERT enables its users (e.g. experts in NLP and linguistics) to (1) interactively explore the fine-tuning process across different model-task pairs, (2) visualize the shape of embedding spaces at multiple scales and layers, and (3) connect linguistic and contextual information about the input dataset with the topology of the embedding space. Using TopoBERT, we provide various use cases to exemplify its applications in exploring fine-tuned word embeddings. We further demonstrate the utility of TopoBERT, which enables users to generate insights about the fine-tuning process and provides support for empirical validation of these insights.

show abstract

Acceptability Judgements via Examining the Topology of Attention Maps

Cited by 2 publications

References 0 publications

Raising the Bar on Acceptability Judgments Classification: An Experiment on ItaCoLA Using ELECTRA

Raising the Bar on Acceptability Judgments Classification: An Experiment on ItaCoLA Using ELECTRA

TopoBERT: Exploring the topology of fine-tuned word representations

Contact Info

Product

Resources

About