The Impact of Word Representations on Sequential Neural MWE Identification

Abstract

Recent initiatives such as the PARSEME shared task have allowed the rapid development of MWE identification systems. Many of those are based on recent NLP advances, using neural sequence models that take continuous word representations as input. We study two related questions in neural verbal MWE identification: (a) the use of lemmas and/or surface forms as input features, and (b) the use of word-based or character-based embeddings to represent them. Our experiments on Basque, French, and Polish show that character-based representations yield systematically better results than word-based ones. In some cases, character-based representations of surface forms can be used as a proxy for lemmas, depending on the morphological complexity of the language.

A Survey of MWE Identification Experiments: The Devil is in the Details

“…Figure 1 shows the distribution of papers across the 24 languages considered by our paper sample. The reasons that lead to choosing a given corpus and/or set of languages in non-ST works are various: language diversity (Zampieri et al, 2019), corpus domain (Liu et al, 2021), and corpus quality and size (Pasquer et al, 2020b).…”

“…6 Most papers do not explicitly mention their strategy to deal with overlapping MWEs. When mentioned, overlapping MWE annotations are either ignored (Zampieri et al, 2022a), duplicated into separate sentences (Zampieri et al, 2018), or handled by the tagging scheme (Yirmibeşoglu and Güngör, 2020).…”

“…Analyses tended to take one of two forms: example-based analysis reporting individual instances where the model performed better or worse than usual (Klyueva et al, 2017;Walsh et al, 2022), and automatic metrics aggregated across particular properties or phenomena. Among the focused metrics, some papers pay special attention to discontinuities (Björne and Salakoski, 2016;Moreau et al, 2018;Berk et al, 2018a;Rohanian et al, 2019) and seen/unseen MWEs (Maldonado et al, 2017;Zampieri et al, 2018;Taslimipoor and Rohanian, 2018). Some studies analyse the model's features and modules via ablation experiments (Scherbakov et al, 2016;Tang et al, 2016;Stodden et al, 2018;Pasquer et al, 2020a).…”

“…Crosslanguage performance was also discussed, especially in the first editions of PARSEME (Simkó et al, 2017;Boros et al, 2017). More original aspects discussed less often include POS sequence patterns (Cordeiro et al, 2016;Tang et al, 2016), the use of external lexicons (Kirilin et al, 2016), syntactic dependencies between components (Pasquer et al, 2018;Moreau et al, 2018), pre-trained embedding representations (Zampieri et al, 2019), and tagging schemes, as discussed in § 4 (Zampieri et al, 2022b).…”

Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023)

“…Figure 1 shows the distribution of papers across the 24 languages considered by our paper sample. The reasons that lead to choosing a given corpus and/or set of languages in non-ST works are various: language diversity (Zampieri et al, 2019), corpus domain , and corpus quality and size (Pasquer et al, 2020b).…”