Effects of using examples on structural model comprehension: a controlled experiment

Abstract

We present a controlled experiment for the empirical evaluation of Example-Driven Modeling (EDM), an approach that systematically uses examples for model comprehension and domain knowledge transfer. We conducted the experiment with 26 graduate and undergraduate students from electrical and computer engineering (ECE), computer science (CS), and software engineering (SE) programs at the University of Waterloo. The experiment involves a domain model, with UML class diagrams representing the domain abstractions and UML object diagrams representing examples of using these abstractions. The goal is to provide empirical evidence of the effects of suitable examples in model comprehension, compared to having model abstractions only, by having the participants perform model comprehension tasks. Our results show that EDM is superior to having model abstractions only, with an improvement of 39% for diagram completeness, 30% for questions completeness, 71% for efficiency, and a reduction of 80% for the number of mistakes. We provide qualitative results showing that participants receiving model abstractions augmented with examples experienced lower perceived difficulty in performing the comprehension tasks, higher perceived confidence in their tasks' solutions, and asked fewer clarifying domain questions, a reduction of 90%. We also present participants' feedback regarding the usefulness of the provided examples, their number and types, as well as, the use of partial examples.

Using Examples for Teaching Software Design: An Experiment Using a Repository of UML Class Diagrams

“…Bkak Many empirical studies were conducted to evaluate comprehension techniques in UML. Zayan et al [8] constructed a controlled experiment for empirical evaluation of EMD. They represented the abstraction part as UML class diagram and the examples of object diagrams.…”

“…Third, the students answered a post-experiment questionnaire. The questionnaire contained 12 questions: seven questions with scale (-4,4) or (1,8), four open questions, and one multiple choice. Finally, the experts used an online evaluation form.…”

An Empirical Investigation to Understand the Difficulties and Challenges of Software Modellers When Using Modelling Tools

“…As per challenges of modelling, few works aim at understanding the challenges of model comprehension. For instance, Kuzniarz et al [23] and Zayana et al [48] investigate the effectiveness of using stereotypes and examples in improving the comprehension of UML models, respectively. Kutar et al [22] test whether users' understanding of information in Sequence diagrams differ from their understanding of Collaboration diagrams.…”

Em direção ao desenvolvimento de um chatbot baseado em exemplos para prática de conteúdos de programação

“…A ABE utiliza a demonstração de como realizar uma determinada tarefa ou habilidade para que, com base na observação, o discente possa aprender determinado conteúdo [20]. Kopp et al [9] argumentam que os discentes abstraem e resolvem problemas mais facilmente se o novo material a ser ensinado for apresentado com vários exemplos (e.g., corretos e errôneos), reduzindo também o esforço mental necessário para compreender os problemas [23].…”