Formative User-Centered Evaluation of Security Modeling

Abstract

Developing a security modeling language is a complex activity. Particularly, it becomes very challenging for Security Requirements Engineering (SRE) languages where social/organizational concepts are used to represent high-level business aspects, while security aspects are typically expressed in a technical jargon at a lower level of abstraction. In order to reduce this socio-technical mismatch and reach a high quality outcome, appropriate evaluation techniques need to be chosen and carried out throughout the development process of the modeling language. In this article, the authors present and discuss the formative user-centered evaluation approach, namely an evaluation technique that starts since the early design stages and actively involves end-users. The authors demonstrate the approach in a real case study presenting the results of the evaluation. From the gained empirical evidence, we may conclude that formative user-centered evaluation is highly recommended to investigate any security modeling language.

Schooladvise: Designing a Reputation System for Educational Services through Service Design and Business Modeling

“…Users are all stakeholders who interacted with the modelling language and tool: two programmers, two software analysts, around 20 stakeholders from nurseries' staff/owners/managers. Following [5] users' experience was gathered by the use of forms and by observing their interaction during the use of the model. Forms.…”

“…Following [5] we evaluated our modeling language for reputation system development focusing on the following points: modelling language a usability, detection of missing concepts, usability of graphical representation, concepts' semantics clarity. The usability criteria used in or evaluation are: effectiveness which measures how users are able to achieve their specified goals through design, efficiency which focuses on the effort made by users to learn and use the modelling language, finally satisfaction checks the overall impression collection feedbacks by users.…”

“…The aim of the analysis presented in this paper is to gather feedback on the proposed model from the endusers (requirements engineers and domain experts) through the use of a practical case. The motivation for this analysis is the following: we are currently developing a visual editor to use the proposed modelling language (by extending VDML and ServiceML meta-models on Eclipse Modelling Framework [11]); following [5] we used a formative evaluation to identify problems in the design tool during its development process. We then followed a user-centred design approach focusing on the adequacy of the modelling features for expressing reputation systems concepts as well as the usability of the modelling language through the application on a practical case.…”

Specifying and Reasoning over Socio-Technical Security Requirements with STS-Tool

“…This version of the tool is the result of an iterative development process, having been tested on multiple case studies and evaluated with practitioners [4] in the scope of the ;'#6$<(/5: 9')073 …”

STS-tool: Socio-technical Security Requirements through social commitments

“…Our work on the STS-ml and tool is ongoing as part of the European research project Aniketos 2 . The current version of the tool is a result of an iterative development process, where the release of internal versions of the tool has been followed by evaluation activities [6].…”