An Empirical Investigation to Understand the Difficulties and Challenges of Software Modellers When Using Modelling Tools

Abstract

Software modelling is a challenging and error-prone task. Existing Model-Driven Engineering (MDE) tools provide modellers with little aid, partly because tool providers have not investigated users' difficulties through empirical investigations such as field studies. This paper presents the results of a two-phase user study to identify the most prominent difficulties that users might face when developing UML Class and State-Machine diagrams using UML modelling tools. In the first phase, we identified the preliminary modelling challenges by analysing 30 Class and State-Machine models that were previously developed by students as a course assignment. The result of the first phase helped us design the second phase of our user study where we empirically investigated different aspects of using modelling tools: the tools' effectiveness, users' efficiency, users' satisfaction, the gap between users' expectation and experience, and users' cognitive difficulties. Our results suggest that users' greatest difficulties are in (1) remembering contextual information and (2) identifying and fixing errors and inconsistencies.

Single-state state machines in model-driven software engineering: an exploratory study

“…Therefore, the authors call for more empirical evidence on MDE subjects to help researchers understand MDE adoption, practice, and experience. Since then, many empirical MDE studies have been conducted to understand how MDE is being adopted and applied in practice (Hutchinson et al 2011;Whittle et al 2013a;Hutchinson et al 2014;Whittle et al 2013b;Farias et al 2013;Pourali and Atlee 2018;Chaudron et al 2012;Liebel et al 2014;Mohagheghi et al 2013). These papers explored different dimensions of MDE adoption and practice, using mostly interviews and surveys.…”

A Focus+Context Approach to Alleviate Cognitive Challenges of Editing and Debugging UML Models

“…Based on the results of our previous empirical study [3], users' most-severe difficulties are: (1) Context: remembering contextual information and (2) Debugging: locating, understanding and fixing errors and inconsistencies in models. To overcome these difficulties, we propose two Focus+Context [4] interfaces that aim to reduce cognitive challenges of developing models by providing users with the information (Context) that are relevant to performing a particular task (Focus).…”

“…In spite of the invaluable benefits offered by the UML, researchers have shown that users' challenges with UML tools are one of the major obstacles to the adoption of MDE by industry [1] [2]. We [3] investigated further and conducted a formative user study to identify and understand the difficulties of editing and debugging UML Class and State-Machine diagrams, representing the static and dynamic diagrams 1 used most permanently in model analysis and code generation, respectively. Their study revealed that users' most-severe cognitive challenges are 1) Context: remembering contextual information needed to write correct, complete, and precise transition expressions in State-Machine diagrams, and 2) Debugging: locating and fixing errors in the model.…”

UCAnDoModels: A Context-Based Model Editor for Editing and Debugging UML Class and State-Machine Diagrams

“…Our prior formative study [15] identified information overload as a major difficulty that modellers face when using model editors. Modelling languages propose different diagrams to model crosscutting concerns of a system.…”

“…Modellers expect model editors to assist in dealing with the relevant information that are separated into different diagrams; but the tools mostly offer navigability features rather than understandability features [10]. According to our formative study [15], an effective model editor should provide features to help users with their challenges of remembering and editing the relevant contextual information (referred to as Context) when performing editing and debugging tasks (referred to as Focus).…”