UPB at SemEval-2021 Task 1: Combining Deep Learning and Hand-Crafted Features for Lexical Complexity Prediction

Zaharia, George-Eduard; Cercel, Dumitru-Clementin; Dascălu, Mihai

doi:10.18653/v1/2021.semeval-1.77

Cited by 4 publications

(2 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, several experiments are performed by combining the 23 linguistic features with the word and sentence embeddings from pre-trained and fine-tuned deep learning models, as in the work done by Liebeskind et al [20]. Therefore, the embeddings at the sentence level from which the token comes and the embeddings at the word level are obtained, as the information from the context of the token constitutes important help so that the embeddings at the word level can be adequate for LCP [21].…”

Section: Featuresmentioning

confidence: 99%

Combining Transformer Embeddings with Linguistic Features for Complex Word Identification

2022

View full text Add to dashboard Cite

Identifying which words present in a text may be difficult to understand by common readers is a well-known subtask in text complexity analysis. The advent of deep language models has also established the new state-of-the-art in this task by means of end-to-end semi-supervised (pre-trained) and downstream training of, mainly, transformer-based neural networks. Nevertheless, the usefulness of traditional linguistic features in combination with neural encodings is worth exploring, as the computational cost needed for training and running such networks is becoming more and more relevant with energy-saving constraints. This study explores lexical complexity prediction (LCP) by combining pre-trained and adjusted transformer networks with different types of traditional linguistic features. We apply these features over classical machine learning classifiers. Our best results are obtained by applying Support Vector Machines on an English corpus in an LCP task solved as a regression problem. The results show that linguistic features can be useful in LCP tasks and may improve the performance of deep learning systems.

show abstract

Section: Featuresmentioning

confidence: 99%

Combining Transformer Embeddings with Linguistic Features for Complex Word Identification

2022

View full text Add to dashboard Cite

show abstract

“…Almeida et al (2021) employed the usage of neural network solutions; more specifically, they used chunks of the sentences obtained with Sent2Vec as input features. Zaharia et al (2021) created models that are based on target and context feature extractors, alongside features resulted from Graph Convolutional Networks, Capsule Networks, and pre-trained word embeddings.…”

Section: Lcp 2021 Complex Datasetmentioning

confidence: 99%

Domain Adaptation in Multilingual and Multi-Domain Monolingual Settings for Complex Word Identification

Zaharia¹,

Smădu²,

Cercel³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Complex word identification (CWI) is a cornerstone process towards proper text simplification. CWI is highly dependent on context, whereas its difficulty is augmented by the scarcity of available datasets which vary greatly in terms of domains and languages. As such, it becomes increasingly more difficult to develop a robust model that generalizes across a wide array of input examples. In this paper, we propose a novel training technique for the CWI task based on domain adaptation to improve the target character and context representations. This technique addresses the problem of working with multiple domains, inasmuch as it creates a way of smoothing the differences between the explored datasets. Moreover, we also propose a similar auxiliary task, namely text simplification, that can be used to complement lexical complexity prediction. Our model obtains a boost of up to 2.42% in terms of Pearson Correlation Coefficients in contrast to vanilla training techniques, when considering the CompLex from the Lexical Complexity Prediction 2021 dataset. At the same time, we obtain an increase of 3% in Pearson scores, while considering a cross-lingual setup relying on the Complex Word Identification 2018 dataset. In addition, our model yields state-ofthe-art results in terms of Mean Absolute Error.

show abstract

Lexical Complexity Prediction: An Overview

2023

View full text Add to dashboard Cite

The occurrence of unknown words in texts significantly hinders reading comprehension. To improve accessibility for specific target populations, computational modelling has been applied to identify complex words in texts and substitute them for simpler alternatives. In this paper, we present an overview of computational approaches to lexical complexity prediction focusing on the work carried out on English data. We survey relevant approaches to this problem which include traditional machine learning classifiers (e.g. SVMs, logistic regression) and deep neural networks as well as a variety of features, such as those inspired by literature in psycholinguistics as well as word frequency, word length, and many others. Furthermore, we introduce readers to past competitions and available datasets created on this topic. Finally, we include brief sections on applications of lexical complexity prediction, such as readability and text simplification, together with related studies on languages other than English.

show abstract

UPB at SemEval-2021 Task 1: Combining Deep Learning and Hand-Crafted Features for Lexical Complexity Prediction

Cited by 4 publications

References 23 publications

Combining Transformer Embeddings with Linguistic Features for Complex Word Identification

Combining Transformer Embeddings with Linguistic Features for Complex Word Identification

Domain Adaptation in Multilingual and Multi-Domain Monolingual Settings for Complex Word Identification

Lexical Complexity Prediction: An Overview

Contact Info

Product

Resources

About