Learning from Mistakes: An Empirical Study of Elicitation Interviews Performed by Novices

Abstract

Interviews are the most widely used elicitation technique in requirements engineering. However, conducting effective requirements elicitation interviews is challenging, due to the combination of technical and soft skills that requirements analysts often acquire after a long period of professional practice. Empirical evidence about training the novices on conducting effective requirements elicitation interviews is scarce. [Objectives] We present a list of most common mistakes that novices make in requirements elicitation interviews. The objective is to assist the educators in teaching interviewing skills to student analysts. [Research Method] We conducted an empirical study involving roleplaying and authentic assessment with 110 students, teamed up in 28 groups, to conduct interviews with a customer. One researcher made observation notes during the interview while two researchers reviewed the recordings. We qualitatively analyzed the data to identify the themes and classify the mistakes. [Results and conclusion] We identified 34 unique mistakes classified into 7 high level themes. We also give examples of the mistakes made by the novices in each theme, to assist the educationists and trainers. Our research design is a novel combination of well-known pedagogical approaches described in sufficient details to make it repeatable for future requirements engineering education and training research.

An Intuitionistic Fuzzy Based Approach to Resolve Detected Ambiguities in the User Requirements Document

“…Hence, it is crucial to provide analysts with cognitive tools to identify and alleviate ambiguities [15]. The requirement elicitor has to put extra effort into pushing the customer to the limit of their tacit knowledge [21]. This statement may seem entirely unambiguous.…”

“…The stakeholders involved in a project may be from different backgrounds, knowledge, regions, or departments; they may have a different understanding of the language. This leads to a prevalent problem of terminological discrepancies [15], [19], [21]. This happens when different users interpret the same word differently based on their understanding and interest.…”

“…Hence, an expert elicitor is required to handle these uncertainties among stakeholders. Another critical reason for ambiguities, as mentioned by several researchers, is the lack of context knowledge that leads to hesitation or terminological discrepancies among the stakeholders and Elicitor [2], [15], [19], [21], [25]. However, context knowledge tools such as WordNet help elicitors make mature decisions to some extent.…”

LadderBot: A Requirements Self-Elicitation System

“…So far, RE literature rarely focuses on characteristics of novice users to be supported in elicitation processes [1]. Commonly, novice RE analysts are the focus of supporting activities [18]. However, insights from analyzing the behavior of novice analysts in elicitation processes may serve as a guideline for how to provide appropriate support for requirements self-elicitation.…”

“…Without uncovering the cause of, or foundation for user needs, the development of disruptive solutions stagnates. We can avoid common mistakes of novice analysts that happen during interviews, such as question formulation, ordering, and question omission through a predefined interview structure [18]. Furthermore, the analyst's behavior, such as lack of confidence or unprofessionalism, or inadequate time management, has a substantial impact on the results of an interview [21].…”