Requirements Classification with Interpretable Machine Learning and Dependency Parsing

Dalpiaz, Fabiano; Dell’Anna, Davide; Aydemir, Fatma Başak; Çevikol, Sercan

doi:10.1109/re.2019.00025

Cited by 44 publications

(46 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kurtanović and Maalej (2017a) additionally use syntactic criteria obtained from constituency and dependency parsing for distinguishing functional and non-functional requirements and further classifying non-functional requirements into sub-categories. Dalpiaz et al (2019) further build upon the findings of Kurtanović and Maalej (2017a) to perform functional and non-functional requirements classification using dependency parsing and lists of commonly-used verbs. Our combination of token-based, frequency-based and syntactic features as well as the use of these features in tandem with semantic ones is novel.…”

Section: Related Workmentioning

confidence: 92%

“…ML has been utilized as a way to provide computerized assistance for several requirements engineering tasks, e.g., trace link generation (Asuncion et al 2010;Cleland-Huang et al 2010;Sultanov and Hayes 2013;Guo et al 2017;Wang et al 2019), requirements identification and classification (Cleland-Huang et al 2007;Winkler and Vogelsang 2016;Kurtanović and Maalej 2017a;Dalpiaz et al 2019;Winkler et al 2019), prioritization (Perini et al 2013), ambiguity detection (Yang et al 2010;Yang et al 2012), relevance analysis (Arora et al 2019), and review classification (Maalej et al 2016;Kurtanovic and Maalej 2017b). The application of ML over textual requirements is almost always preceded by some form of NLP.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Automated demarcation of requirements in textual specifications: a machine learning-based approach

et al. 2020

View full text Add to dashboard Cite

A simple but important task during the analysis of a textual requirements specification is to determine which statements in the specification represent requirements. In principle, by following suitable writing and markup conventions, one can provide an immediate and unequivocal demarcation of requirements at the time a specification is being developed. However, neither the presence nor a fully accurate enforcement of such conventions is guaranteed. The result is that, in many practical situations, analysts end up resorting to after-the-fact reviews for sifting requirements from other material in a requirements specification. This is both tedious and time-consuming. We propose an automated approach for demarcating requirements in free-form requirements specifications. The approach, which is based on machine learning, can be applied to a wide variety of specifications in different domains and with different writing styles. We train and evaluate our approach over an independently labeled dataset comprised of 33 industrial requirements specifications. Over this dataset, our approach yields an average precision of 81.2% and an average recall of 95.7%. Compared to simple baselines that demarcate requirements based on the presence of modal verbs and identifiers, our approach leads to an average gain of 16.4% in precision and 25.5% in recall. We collect and analyze expert feedback on the demarcations produced by our approach for industrial requirements specifications. The results indicate that experts find our approach useful and efficient in practice. We developed a prototype tool, named DemaRQ, in support of our approach. To facilitate replication, we make available to the research community this prototype tool alongside the non-proprietary portion of our training data.

show abstract

Section: Related Workmentioning

confidence: 92%

Section: Related Workmentioning

confidence: 99%

Automated demarcation of requirements in textual specifications: a machine learning-based approach

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Ambiguity has been widely studied in the requirements engineering (RE) literature [3], [8]- [11]. Both manual ap proaches based on reviews and inspections [9], [12], and automated approaches based on natural language processing (NLP) [6], [7], [13], [14], have been proposed for detecting ambiguity in requirements. Some recent works use domainspecific corpora for detecting terms that are likely to be ambiguous due to different meanings across domains [6], [15]- [17].…”

Section: Introductionmentioning

confidence: 99%

Using Domain-Specific Corpora for Improved Handling of Ambiguity in Requirements

Ezzini

Abualhaija

Arora

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Ambiguity in natural-language requirements is a pervasive issue that has been studied by the requirements en gineering community for more than two decades. A fully manual approach for addressing ambiguity in requirements is tedious and time-consuming, and may further overlook unacknowledged ambiguity -the situation where different stakeholders perceive a requirement as unambiguous but, in reality, interpret the requirement differently. In this paper, we propose an automated approach that uses natural language processing for handling ambiguity in requirements. Our approach is based on the au tomatic generation of a domain-specific corpus from Wikipedia. Integrating domain knowledge, as we show in our evaluation, leads to a significant positive improvement in the accuracy of ambiguity detection and interpretation. We scope our work to coordination ambiguity (CA) and prepositional-phrase attach ment ambiguity (PAA) because of the prevalence of these types of ambiguity in natural-language requirements [1]. We evaluate our approach on 20 industrial requirements documents. These documents collectively contain more than 5000 requirements from seven distinct application domains. Over this dataset, our approach detects CA and PAA with an average precision of ~80% and an average recall of ~89% (~90% for cases of unac knowledged ambiguity). The automatic interpretations that our approach yields have an average accuracy of ~85%. Compared to baselines that use generic corpora, our approach, which uses domain-specific corpora, has ~33% better accuracy in ambiguity detection and ~16% better accuracy in interpretation.

show abstract

“…Normally software requirements are of two types, namely, FRs and NFRs. e research work on the difference between FRs and NFRs is defined and well known; however, the automatic identification and classification of the software requirements stated in different natural language is still a huge challenge [14][15][16][17][18][19][20][21].…”

Section: Introductionmentioning

confidence: 99%

A Systematic Literature Review on Using Machine Learning Algorithms for Software Requirements Identification on Stack Overflow

Ahmad

Feng

Khan

et al. 2020

Security and Communication Networks

View full text Add to dashboard Cite

Context. The improvements made in the last couple of decades in the requirements engineering (RE) processes and methods have witnessed a rapid rise in effectively using diverse machine learning (ML) techniques to resolve several multifaceted RE issues. One such challenging issue is the effective identification and classification of the software requirements on Stack Overflow (SO) for building quality systems. The appropriateness of ML-based techniques to tackle this issue has revealed quite substantial results, much effective than those produced by the usual available natural language processing (NLP) techniques. Nonetheless, a complete, systematic, and detailed comprehension of these ML based techniques is considerably scarce. Objective. To identify or recognize and classify the kinds of ML algorithms used for software requirements identification primarily on SO. Method. This paper reports a systematic literature review (SLR) collecting empirical evidence published up to May 2020. Results. This SLR study found 2,484 published papers related to RE and SO. The data extraction process of the SLR showed that (1) Latent Dirichlet Allocation (LDA) topic modeling is among the widely used ML algorithm in the selected studies and (2) precision and recall are amongst the most commonly utilized evaluation methods for measuring the performance of these ML algorithms. Conclusion. Our SLR study revealed that while ML algorithms have phenomenal capabilities of identifying the software requirements on SO, they still are confronted with various open problems/issues that will eventually limit their practical applications and performances. Our SLR study calls for the need of close collaboration venture between the RE and ML communities/researchers to handle the open issues confronted in the development of some real world machine learning-based quality systems.

show abstract

Requirements Classification with Interpretable Machine Learning and Dependency Parsing

Cited by 44 publications

References 35 publications

Automated demarcation of requirements in textual specifications: a machine learning-based approach

Automated demarcation of requirements in textual specifications: a machine learning-based approach

Using Domain-Specific Corpora for Improved Handling of Ambiguity in Requirements

A Systematic Literature Review on Using Machine Learning Algorithms for Software Requirements Identification on Stack Overflow

Contact Info

Product

Resources

About