Towards Model Quality Assurance for Multi-Disciplinary Engineering

Winkler, Dietmar; Wimmer, Manuel; Berardinelli, Luca; Biffl, Stefan

doi:10.1007/978-3-319-56345-9_16

Cited by 6 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although traditional software inspection has many benefits for defect detection, it also faces limitations such as: (i) lack of experts, which are expensive and scarce resources, to conduct defect detection and validation; (ii) limited scalability of inspection processes typically scheduled for two hours thus constraining the scope of the inspection artifacts; (iii) challenging inspection control and coordination, due to lack of suitable tool support which leads to inspection processes being planned and executed without in-process feedback, e.g., for clarifications or coordination (Winkler et al 2017). HC techniques could help to overcome these limitations by involving groups of junior software developers working off-site, scaling up the process by distributing the tasks over a larger expert base and by providing tooling for in-process control and improved coordination.…”

Section: Eer Diagram Verificationmentioning

confidence: 99%

“…Therefore, the verification of (EER) models is key to mitigate the risk of high rework. Yet, current EER verification processes performed on-site by expert groups suffer from limited expert availability, low scalability and lack of tool support for advanced process control (Winkler et al 2017). HC techniques could address these issues by scaling up the process through distribution to a larger base of junior software developers working off-site through online, distributed work.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Verifying Extended Entity Relationship Diagrams with Open Tasks

Sabou

Käsznar

Zlabinger

et al. 2020

HCOMP

View full text Add to dashboard Cite

The verification of Extended Entity Relationship (EER) diagrams and other conceptual models that capture the design of information systems is crucial to ensure reliable systems. To scale up verification processes to larger groups of experts, Human Computation techniques were used focusing primarily on closed tasks, which constrain the number and variety of reported defects in favor of easy aggregation of derived judgements. To address this limitation of closed tasks, in this paper, we investigate EER verification (as instance of a broader family of model verification problems) with open tasks to extend the range of collected results. We also address the challenge of aggregating results of open tasks by proposing a follow-up HC task for defect validation. We evaluate our approach for HC-based EER Verification with open tasks in a set of experiments conducted with junior developers and show that (1) open tasks allow collecting a variety of insights that go beyond a manually built gold standard while still leading to good performance (F1=60%) and (2) HC-based validation can be reliably used for validating the results of open tasks (F1=84% compared to expert validation).

show abstract

Section: Eer Diagram Verificationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Verifying Extended Entity Relationship Diagrams with Open Tasks

Sabou

Käsznar

Zlabinger

et al. 2020

HCOMP

View full text Add to dashboard Cite

show abstract

“…Based on the provided focus on deviations and changes, the "Focused Inspection" [45] and "Model Quality Assurance" approaches [46] help to improve the unguided defect detection process by supporting (a) the review team selection and (b) their reading process. Individual engineers focus on defined aspects of the systems, for example, on mechanical, electrical, or software characteristics of the engineering plan, and their overlaps with partner disciplines.…”

Section: Common Concepts With Limited Semantic Web Technologies (Levementioning

confidence: 99%

Improving Quality Assurance in Multidisciplinary Engineering Environments with Semantic Technologies

Winkler¹,

Sabou²,

Biffl³

2017

Quality Control and Assurance - An Ancient Greek Term Re-Mastered

Self Cite

View full text Add to dashboard Cite

In multidisciplinary engineering (MDE) projects, for example, automation systems or manufacturing systems, stakeholders from various disciplines, for example, electrics, mechanics and software, have to collaborate. In industry practice, engineers apply individual and highly specialized tools with strong limitation regarding defect detection in early engineering phases. Experts typically execute reviews with limited tool support which make engineering projects defective and risky. Semantic Web Technologies (SWTs) can help to bridge the gap between heterogeneous sources as foundation for efficient and effective defect detection. Main questions focus on (a) how to bridge gaps between loosely coupled tools and incompatible data models and (b) how SWTs can help to support efficient and effective defect detection in context of engineering process improvement. This chapter describes success-critical requirements for defect detection in MDE and shows how SWTs can provide the foundation for early and efficient defect detection with an adapted review approach. The proposed defect detection framework (DDF) suggests different levels of SWT contributions as a roadmap for engineering process improvement. Two selected industry-related real-life cases show different levels of SWT involvement. Although SWTs have been successfully applied in real-life use cases, SWT applications can be risky if applied without good understanding of success factors and limitations.

show abstract

“…However, the manual verification of conceptual models by experts is time-consuming, costly and faces several challenges even when following well-established procedures such as in software inspection, e.g., difficulty to cover the entire inspection object during typical sessions of two hours, high costs and time needed for coordinating the inspection group without dedicated tool support (Winkler et al 2017c). In this context, human computation techniques promise, at a minimum, tool support and a better coordination of the model verification process as well as its outsourcing to crowds of layman or semi-experts.…”

Section: Introductionmentioning

confidence: 99%

Verifying Conceptual Domain Models with Human Computation: A Case Study in Software Engineering

Sabou

Winkler

Penzerstadler

et al. 2018

HCOMP

View full text Add to dashboard Cite

Conceptual domain models, such as taxonomies, knowledge graphs or Extended Entity Relationship (EER) diagrams are core to all information systems. The task of verifying the correctness of these models is of high interest to the knowledge and software engineering communities and attracted the first solution approaches using human computation. Yet, since these solutions are published within the boundaries of their communities, there is a lack of concerted work on this topic. As a first step to alleviate this status quo, we formalize the problem of verifying conceptual models and propose a generic approach (VeriCoM) to solve it with human computation techniques. We show how VeriCoM was applied in a software engineering use case focusing on verifying the correctness of an EER diagram against a system specification document. An evaluation of VeriCoM in a series of four workshops within one controlled experiment performed with a crowd of semi-experts lead to the identification of a set of defects with precision of 73% and a recall from a Gold Standard defect set of 63%.

show abstract

Towards Model Quality Assurance for Multi-Disciplinary Engineering

Cited by 6 publications

References 18 publications

Verifying Extended Entity Relationship Diagrams with Open Tasks

Verifying Extended Entity Relationship Diagrams with Open Tasks

Improving Quality Assurance in Multidisciplinary Engineering Environments with Semantic Technologies

Verifying Conceptual Domain Models with Human Computation: A Case Study in Software Engineering

Contact Info

Product

Resources

About