Crowd-Generated Data Mining for Continuous Requirements Elicitation

Abstract

In software development projects, the process of requirements engineering (RE) is one in which requirements are elicited, analyzed, documented, and managed. Requirements are traditionally collected using manual approaches, including interviews, surveys, and workshops. Employing traditional RE methods to engage a large base of users has always been a challenge, especially when the process involves users beyond the organization's reach. Furthermore, emerging software paradigms, such as mobile computing, social networks, and cloud computing, require better automated or semi-automated approaches for requirements elicitation because of the growth in systems users, the accessibility to crowd-generated data, and the rapid change of users' requirements. This research proposes a methodology to capture and analyze crowd-generated data (e.g., user feedback and comments) to find potential requirements for a software system in use. It semi-automates some requirementselicitation tasks using data retrieval and natural language processing (NLP) techniques to extract potential requirements. It supports requirements engineers' efforts to gather potential requirements from crowd-generated data on social networks (e.g., Twitter). It is an assistive approach that taps into unused knowledge and experiences emphasizing continuous requirements elicitation during systems use.

Research Efforts and Challenges in Crowd-based Requirements Engineering: A Review

“…All of the crowd-based RE efforts require the users' involvement to obtain information. iRequire [17], AppEcho [19], FAME [21], SUPERSEDE [25], Classification Model [28], Continuous Requirement Elicitation Methodology [29] and Automated Feature Identification [31] fully rely on the users to provide data to developers. Other than that, Crowdsourced RE Platform for User Story (US) Authoring [32] also fully depends on the users' participation.…”

“…Besides, there exist efforts that incorporating Artificial Intelligence techniques to process information obtained from the crowd to produce requirements descriptions such as SUPERSEDE [25], Effective Classification Methodology [11], CREeLS [24], Classification Model [28], Continuous Requirement Elicitation Methodology [29] and Automated Feature Identification [31] to handle reliability issues. This is important because the information gathered could be from anybody who gave their responses and feedback.…”

“…It is also managed to identify relevant information to decide the essential requirements for the next release of a product. In 2019 Continuous Requirement Elicitation Methodology [29] and Automated Feature Identification [31] were proposed. Both of these researches use feedbacks as the primary source for the information.…”

“…Both of these researches use feedbacks as the primary source for the information. A study conducted by A. Alwadin and M. Asharagi in [29] collected feedback and comments from the crowd via Twitter for an in-use software system. They applied data retrieval and natural language processing (NLP) techniques to extract potential requirements.…”

Data-Driven Requirements Elicitation: A Systematic Literature Review

“…• Microblog data from twitter, Facebook, and Weibo were used for automated requirements elicitation. Of total eleven studies that used twitter, four studies extracted only texts [64][65][66][67], while the rest extracted additional metadata [68][69][70][71][72][73][74]. The metadata include the number of retweets, likes, lexically similar tweets (i.e., duplicates), twitter followers and friends (i.e., social rank), replies to tweets, as well as hashtags, handles (i.e., indicated by an @ appended with a username), and demographic data of the person who tweeted.…”