Transformer-based Language models for the Identification of Idiomatic Expressions

Abstract

Idiomatic expressions are part of everyday speech in many languages and text genres. However, identifying idiomatic expressions can be problematic for different NLP tasks as their meaning cannot be easily inferred from their constituting words. Although the automatic identification and understanding of idiomatic expressions are essential for Natural Language Understanding tasks, they are still largely underinvestigated. Until recently, the complex nature and lack of large datasets have prevented the development of machine learning approaches for identifying the idiomatic expressions. With the advancement of machine learning techniques, transformer-based language models such as DistilBERT, RoBERTa, and their variants have shown state of the art performance by capturing the compositionality of the textual representations many NLP tasks. In spite of the progress, these vector representations fail to identify the multiword expressions (MWEs) such as idioms. This study demonstrate that transformer-based language models using contextual embeddings perform much better than existing approaches when applied to texts associated with depression. The core of this article presents two subtasks: (a) binary classification or idiomatic expression identification using BERT sentence embeddings and (b) task based on the analysis of the idiomatic expression usage in depressive textual contents.

Çeviribiliminin Yapay Zeka (YZ) ile Geliştirilmesi: Zorluklar, İmkânlar ve Öneriler

“…AI models often need help with idiomatic expressions, cultural references, and nuances of language, which can lead to inaccurate or inappropriate translations. That highlights the importance of human expertise and the need for human translators to ensure translation accuracy and cultural appropriateness (Nanomi Arachchige et al, 2022).…”