Improving the Usability of a MAS DSML

Miranda, Tomás; Challenger, Moharram; Tezel, Baris Tekin; Alaca, Omer Faruk; Barisic, Ankica; Amaral, Vasco; Goulão, Miguel; Kardaş, Geylani

doi:10.1007/978-3-030-25693-7_4

Cited by 5 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With regard to quality evaluation procedures, the literature in the model-driven engineering field provides examples of the applicability of quality evaluation frameworks in the following: over specific modelling initiatives (e.g. the Physics of Notations -PoN (Moody, 2009) in Big Data analytics (Khalajzadeh et al, 2020) and multiagent (Miranda et al, 2019) modelling languages); specific quality evaluation approaches in specific modelling domains (e.g. the embedded systems domain (Arslan & Kardas, 2020), the multiagent domain (Asici et al, 2021;Alaca et al, 2021)); comparison between modelling initiatives (Santos et al, 2020); and analysis of modelling approaches from the perspective of potential users of the modelling initiative (Shin, 2019).…”

Section: Overview Of Quality Methods For Mdementioning

confidence: 99%

Empirical validation of a quality framework for evaluating modelling languages in MDE environments

et al. 2021

View full text Add to dashboard Cite

In previous research, we proposed the multiple modelling quality evaluation framework (MMQEF), which is a method and tool for evaluating modelling languages in modeldriven engineering (MDE) environments. Rather than being exclusive, MMQEF attempts to complement other methods of evaluation of quality such as SEQUAL. However, to date, MMQEF has not been validated beyond some concept proofs. This paper evaluates the applicability of the MMQEF method in comparison with other existing methods. We performed an evaluation in which the subjects had to detect quality issues in modelling languages. A group of expert professionals and two experimental objects (i.e. two combinations of different modelling languages based on real industrial practices) were used. To analyse the results, we applied quantitative approaches, i.e. statistical tests on the results of the performance measures and the perception of subjects. We ran four replications of the experiment in Colombia between 2016 and 2019, with a total of 50 professionals. The results of the quantitative analysis show a low performance for all of the methods, but a positive perception of MMQEF.Conclusions: The application of modelling language quality evaluation methods within MDE settings is indeed tricky, and subjects did not succeed in identifying all quality problems. This experiment paves the way for additional investigation on the trade-offs between the methods and potential situational guidelines (i.e. circumstances under which each method is convenient). We encourage further inquiries on industrial applications to incrementally improve the method and tailor it to the needs of professionals working in real industrial environments.

show abstract

Section: Overview Of Quality Methods For Mdementioning

confidence: 99%

Empirical validation of a quality framework for evaluating modelling languages in MDE environments

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Second, a single evaluator group was used instead of two different groups, which could pose a threat to the execution phase. We experienced using both single and double evaluator groups in our previous empirical studies on evaluating both different MAS DSMLs (e.g., Kardas et al, 2017;Miranda et al, 2019) and DSMLs in other industrial domains (e.g., Saritas and Kardas, 2014;Arslan and Kardas, 2020). Using a single group may raise the risk that the evaluators take advantage of their prior development experience using the MAS DSML while developing the same MAS without using this MAS DSML (or vice-versa).…”

Section: Threats To the Validitymentioning

confidence: 99%

“…Randomizing the order of the evaluator groups and/or the applied case studies can be an option. For instance, we followed such a randomization technique on evaluating the usability of the syntax of a MAS DSML in (Miranda et al, 2019) where evaluator groups and MAS development case studies are randomized.…”

Section: Threats To the Validitymentioning

confidence: 99%

“…However, we could not follow the same approach here due to: 1) the enormous size of the case studies, complete implementation of two complex MAS from scratch instead of just modeling as is the case in (Miranda et al, 2019); 2) time limitations for the experiments; and 3) the unavailability of all evaluators for such an extensive repeating model of evaluation.…”

Section: Threats To the Validitymentioning

confidence: 99%

“…Finally, in our recent work (Silva et al, 2018;Miranda et al, 2019), we investigate how the graphical syntax of MAS DSMLs can be improved by applying the principles of the "Physics"…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations