Requirements Engineering Challenges in Building AI-Based Complex Systems

Abstract

This paper identifies and tackles the challenges of the requirements engineering discipline when applied to development of AI-based complex systems. Due to their complex behaviour, there is an immanent need for a tailored development process for such systems. However, there is still no widely used and specifically tailored process in place to effectively and efficiently deal with requirements suitable for specifying a software solution that uses machine learning. By analysing the related work from software engineering and artificial intelligence fields, potential contributions have been recognized from agent-based software engineering and goal-oriented requirements engineering research, as well as examples from large product development companies. The challenges have been discussed, with proposals given how and when to tackle them. RE4AI taxonomy has also been outlined, to inform the tailoring of development process.

A Multiple Case Study of Artificial Intelligent System Development in Industry

“…Instead, the expectation of the functional performance of the AI systems evolves over time. Another important part of the systems is non-functional requirements, i.e explainability [28]. However, we find this concern is application domain-specific.…”

“…Some challenges are revealed, i.e. undefined processes [7], communication challenges [16], team competences [10] and effort estimation [28]. However, these challenges are claimed in a general context, without actionable insights.…”

Construction of a quality model for machine learning systems

“…Although quality properties are being (re)defined for ML components, the literature from the field of requirements engineering shows many challenges when developing and implementing quality models in the context of ML systems (Belani et al, 2019;Bosch et al, 2018;Horkoff, 2019;Ismail et al, 2019;Vogelsang & Borg, 2019). On the one side, data scientists bring new types of requirements (e.g., in terms of data quality).…”

“…Fourth, quality assurance, and specifically testing, works differently than in traditional software. This is because data-driven methods (such as ML, for instance) target problems where the expected solution is inherently difficult to formalize, and where test oracles are not directly available (Belani et al, 2019;Bosch et al, 2018;Horkoff, 2019;Zhang et al, 2020).…”

Pairing conceptual modeling with machine learning

“…Generally, elicitation of business needs and business requirements within the context of ML-based information systems is a novel field of research. However, it is dominated more by questions and challenges, than answers, such as the lack of domain knowledge, undeclared consumers, and unclear problem and scope [142]. Proposed approaches to business intelligence have a strong overlap with ML-based information systems [143].…”