Neural Language Models and Few Shot Learning for Systematic Requirements Processing in MDSE

Bertram, Vincent; Boß, Miriam; Kusmenko, Evgeny; Nachmann, Imke Helene; Rumpe, Bernhard; Trotta, Danilo; Wachtmeister, Louis

doi:10.1145/3567512.3567534

Cited by 4 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…NLP for Requirements Engineering is a growing field with many recent works focusing on requirements classification by fine-tuning pre-trained models [9]- [11]. Further, there have been a few works that focus on building NLP approaches for domain specific language use cases [12]- [17]. Although still a nascent field, there have been a few works in using FSL for requirements classification [18], [19].…”

Section: Introductionmentioning

confidence: 99%

Few-shot learning approaches for classifying low resource domain specific software requirements

Nayak¹,

Timmapathini²,

Murali³

et al. 2023

Preprint

View full text Add to dashboard Cite

With the advent of strong pre-trained natural language processing models like BERT, DeBERTa, MiniLM, T5, the data requirement for industries to fine-tune these models to their niche use cases has drastically reduced (typically to a few hundred annotated samples for achieving a reasonable performance). However, the availability of even a few hundred annotated samples may not always be guaranteed in low resource domains like automotive, which often limits the usage of such deep learning models in an industrial setting. In this paper we aim to address the challenge of fine-tuning such pre-trained models with only a few annotated samples, also known as Few-shot learning. Our experiments focus on evaluating the performance of a diverse set of algorithms and methodologies to achieve the task of classifying BOSCH automotive domain textual software requirements into 3 categories, while utilizing only 15 annotated samples per category for fine-tuning. We find that while SciBERT and DeBERTa based models tend to be the most accurate at 15 training samples, their performance improvement scales minimally as the number of annotated samples is increased to 50 in comparison to Siamese and T5 based models.

show abstract

Section: Introductionmentioning

confidence: 99%

Few-shot learning approaches for classifying low resource domain specific software requirements

Nayak¹,

Timmapathini²,

Murali³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…We discuss threats to validity in Section 8 before Section 9 concludes the paper. This paper is an extended version of the corresponding SLE publication by Bertram et al 2022 [4].…”

Section: Introductionmentioning

confidence: 99%

Technical Report on Neural Language Models and Few-Shot Learning for Systematic Requirements Processing in MDSE

Bertram¹,

Boß²,

Kusmenko³

et al. 2022

Preprint

View full text Add to dashboard Cite

Systems engineering, in particular in the automotive domain, needs to cope with the massively increasing numbers of requirements that arise during the development process. To guarantee a high product quality and make sure that functional safety standards such as ISO26262 are fulfilled, the exploitation of potentials of model-driven systems engineering in the form of automatic analyses, consistency checks, and tracing mechanisms is indispensable. However, the language in which requirements are written, and the tools needed to operate on them, are highly individual and require domainspecific tailoring. This hinders automated processing of requirements as well as the linking of requirements to models. Introducing formal requirement notations in existing projects leads to the challenge of translating masses of requirements and process changes on the one hand and to the necessity of the corresponding training for the requirements engineers.In this paper, based on the analysis of an open-source set of automotive requirements, we derive domain-specific language constructs helping us to avoid ambiguities in requirements and increase the level of formality. The main contribution is the adoption and evaluation of few-shot learning with large pretrained language models for the automated translation of informal requirements to structured languages such as a requirement DSL. We show that support sets of less than ten translation examples can suffice to few-shot train a language model to incorporate keywords and implement syntactic rules into informal natural language requirements.

show abstract

Leveraging Natural Language Processing for a Consistency Checking Toolchain of Automotive Requirements

Bertram,

Kausch,

Kusmenko

et al. 2023

2023 IEEE 31st International Requirements Engineering Conference (RE)

View full text Add to dashboard Cite

Neural Language Models and Few Shot Learning for Systematic Requirements Processing in MDSE

Cited by 4 publications

References 13 publications

Few-shot learning approaches for classifying low resource domain specific software requirements

Few-shot learning approaches for classifying low resource domain specific software requirements

Technical Report on Neural Language Models and Few-Shot Learning for Systematic Requirements Processing in MDSE

Leveraging Natural Language Processing for a Consistency Checking Toolchain of Automotive Requirements

Contact Info

Product

Resources

About