Comparing Model Coverage and Code Coverage in Model Driven Testing: An Exploratory Study

Amalfitano, Domenico; Simone, Vincenzo De; Fasolino, Anna Rita; Riccio, Vincenzo

doi:10.1109/asew.2015.18

Cited by 3 publications

(13 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… Test cases are fully automatically transformed from model into code by using specific industrial or tailor‐made tools. Software specifications are automatically transformed into code, and test cases generated to cover the specification are automatically applied to code without manual intervention [16,17,20,24,31‐33,36,46]. Abstract, model‐level test cases are transformed into concrete, code‐level test cases after stepwise model‐to‐model and model‐to‐code transformations that are performed either automatically or by hand.…”

Section: Analysis Based On the Research Questionsmentioning

confidence: 99%

“…In additional round (from experts), studies are listed in columns related to backward snowballing for convenience. For discussing RQ1, we grouped the 23 studies listed in the first line of Table 4 as follows: studies that directly provided information regarding the transformation of test cases across the abstraction levels by describing the tool that supports the transformation [16,17,20,24,[31][32][33]36,46], studies that described procedures for transforming test cases from models to code [25,34,[38][39][40][41]44,45,47] and studies that just reported that test cases developed at the model level are then applied to test the code [21][22][23]26,43]. Studies from the three groups are discussed in the sequence.…”

Section: Analysis Based On the Research Questionsmentioning

confidence: 99%

“…Camus et al [20] employed the SCADE tool suite 15 to automatically transform model-based test cases to be directly applied to source code. The Model Test Coverage 16 (MTC) tool was employed to run tests and collect model coverage data. They also applied structural code coverage analysis on the code.…”

Section: Analysis Based On the Research Questionsmentioning

confidence: 99%

“…For example, the US FAA requires full code coverage to certify safety critical avionics software [8] and requires that each test must be derived from the requirements. This makes it imperative that when model‐based test cases are transformed to the concrete level, testers are able to ensure traceability from abstract tests to concrete tests [16]. When code is automatically derived from models, potential problems with the transformation m motivate the application of the so‐called witness functions (in the form of traceability [17]) that allow differences to be discovered.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

On transforming model‐based tests into code: A systematic literature review

Ferrari,

Durelli,

Andler

et al. 2023

Software Testing Verif & Rel

View full text Add to dashboard Cite

SummaryModel‐based test design is increasingly being applied in practice and studied in research. Model‐based testing (MBT) exploits abstract models of the software behaviour to generate abstract tests, which are then transformed into concrete tests ready to run on the code. Given that abstract tests are designed to cover models but are run on code (after transformation), the effectiveness of MBT is dependent on whether model coverage also ensures coverage of key functional code. In this article, we investigate how MBT approaches generate tests from model specifications and how the coverage of tests designed strictly based on the model translates to code coverage. We used snowballing to conduct a systematic literature review. We started with three primary studies, which we refer to as the initial seeds. At the end of our search iterations, we analysed 30 studies that helped answer our research questions. More specifically, this article characterizes how test sets generated at the model level are mapped and applied to the source code level, discusses how tests are generated from the model specifications, analyses how the test coverage of models relates to the test coverage of the code when the same test set is executed and identifies the technologies and software development tasks that are on focus in the selected studies. Finally, we identify common characteristics and limitations that impact the research and practice of MBT: (i) some studies did not fully describe how tools transform abstract tests into concrete tests, (ii) some studies overlooked the computational cost of model‐based approaches and (iii) some studies found evidence that bears out a robust correlation between decision coverage at the model level and branch coverage at the code level. We also noted that most primary studies omitted essential details about the experiments.

show abstract

Section: Analysis Based On the Research Questionsmentioning

confidence: 99%

Section: Analysis Based On the Research Questionsmentioning

confidence: 99%

Section: Analysis Based On the Research Questionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On transforming model‐based tests into code: A systematic literature review

Ferrari,

Durelli,

Andler

et al. 2023

Software Testing Verif & Rel

View full text Add to dashboard Cite

show abstract

“…In [10], the author conducted an exploratory study to evaluate the differences that may exist between the model coverage guaranteed by the test scenarios and the code coverage achieved when they were run on the automatically generated code. They focus only on the code coverage, when we also consider the mutation analysis, the complementary tests, and the test creation effort.…”

Section: Related Workmentioning

confidence: 99%

Shall We Test Service-Based Models or Generated Code?

Mottu

André

Coutand

et al. 2019

2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)

View full text Add to dashboard Cite

Testing early reduces the cost of the validation process. Model-Driven Testing promotes the creation of the tests at the model level and their transformation into code which can be executed. However, we miss an empirical evaluation on the efficiency of generating tests designed on the service-based component models compared to implementing them based on the generated code. In this paper, we evaluate if one shall generate tests from platform-independent models or implement them on the generated code. We consider servicebased component models and a case study modeling and generating the code of a platoon of vehicles. Our experimentation on that first case study confirms that generating tests from the service-based models is easier compared to implementing them on the generated code. Moreover, efficiency of the tests of both approaches is similar, considering code coverage and mutation analysis.

show abstract

Towards a Thing-In-the-Loop approach for the Verification and Validation of IoT systems

Amalfitano

Amatucci

Simone

et al. 2017

Proceedings of the 1st ACM Workshop on the Internet of Safe Things

View full text Add to dashboard Cite

The Internet of Things (IoT) is rapidly increasing its diffusion, posing great challenges to the research community. IoT systems are composed by smart objects (Things) that are interconnected in order to provide new products and services. The interaction of heterogeneous and distributed smart things guided by software with the physical world brings new sources of safety issues. To this reason, providing valuable and effective solutions to support the verification and validation of such systems is needed. In this paper we introduce a model-driven Thing-In-the-Loop verification and validation approach that transfers the best practices adopted in different embedded system domains towards the IoT world. Starting from models and scenarios representing the structure and behaviors of the IoT system as well as models of its context our approach generates appropriate test cases that are executed in accordance with Model-in-the-Loop, Software-in-the-Loop and Hardware-in-the-Loop techniques. We preliminarily evaluated the feasibility of our approach by applying it in the context of a Smart Mobility case study

show abstract

Comparing Model Coverage and Code Coverage in Model Driven Testing: An Exploratory Study

Cited by 3 publications

References 5 publications

On transforming model‐based tests into code: A systematic literature review

On transforming model‐based tests into code: A systematic literature review

Shall We Test Service-Based Models or Generated Code?

Towards a Thing-In-the-Loop approach for the Verification and Validation of IoT systems

Contact Info

Product

Resources

About