Exploring Natural Language Processing in Model-To-Model Transformations

Abstract

In this paper, we explore the possibility to apply natural language processing in visual modelto-model (M2M) transformations. Therefore, we present our research results on information extraction from text labels in process models modeled using Business Process Modeling Notation (BPMN) and use case models depicted in Unified Modeling Language (UML) using the most recent developments in natural language processing (NLP). In this paper, we focus on three relevant tasks, namely, the extraction of verb/noun phrases that would be used to form relations, parsing of conjunctive/disjunctive statements, and the detection of abbreviations and acronyms. Relation extraction was attempted to solve by implementing techniques that combine state-of-the-art NLP language models with formal regular expressions grammarbased structure detection. In this paper, we perform thorough testing of the most recent state-of-the-art NLP tools (CoreNLP, Stanford Stanza, Flair, Spacy, AllenNLP, BERT, ELECTRA), as well as custom BERT-BiLSTM-CRF and ELMo-BiLSTM-CRF implementations, trained with certain data augmentations to improve performance on the most ambiguous cases; these tools are used as a foundation for building tools to extract noun and verb phrases from short text labels generally used in UML and BPMN models. Furthermore, we describe our attempts to improve these extractors by solving the abbreviation/acronym detection problem using machine learning-based detection, as well as process conjunctive and disjunctive statements, due to their relevance to performing advanced text normalization. The obtained results show that the best phrase extraction and conjunctive phrase processing performance was obtained using Stanza based implementation, yet, our trained BERT-BiLSTM-CRF outperformed it for the verb phrase detection task. Our acronym detection approach resulted in the precision of 0.78 and F1-Score of 0.73 which may also be considered quite positive. While this work was inspired by our ongoing research on partial model-to-model transformations, we believe it to be applicable in other areas requiring similar text processing capabilities as well. INDEX TERMSinformation extraction; relation extraction; acronym detection; process models; use-case models; natural language processing; model-to-model transformation This work is inspired by our previous research on NLPenhanced information extraction in model-to-model trans- formations [6, 7]. However, the need for similar solutions was also identified in other areas involving visual modeling, such as business process modeling [8,9,10]. In this paper, we address the issue of relation extraction from graphical

Smart Lab: A Non-invasive IoT-centric Approach for Indoor Environment Automation

“…It allows computers to perform various tasks related to natural language at all levels, from syntactic analysis and classification of speech's parts to automatic translation and dialogue systems [43]. Henceforward, machine learning algorithms encompassing NLP and NLU arise, which is the case with the model used [44]. The Wit.ai platform used to implement the NLP and NLU functions supports about 50 languages.…”

Natural Language Processing in Knowledge-Based Support for Operator Assistance

“…Many types of research analyzing NLP tasks consider both traditional and deep learning pre-trained libraries and compare them together in different aspects [52,53]. The transformer-based technique performs efficiently in entity recognition, information extraction, and semantic analysis [54][55][56][57]. However, in the preliminary step of pre-processing, particularly part-ofspeech (POS) tagging, traditional NLP techniques demonstrate notable efficiency [58,59].…”

Evaluating the machine learning models based on natural language processing tasks

“…However, a significant leap in language model performance is achieved when these models are underpinned by neural networks. This integration of neural networks significantly broadens the spectrum of natural language processing (NLP) tasks that a language model can tackle [1]- [3]. A neural language model exhibits versatility in handling NLP tasks, spanning from straightforward to intricate challenges.…”