Model-Based Testing: Achievements and Future Challenges

Mlynarski, Michael; Güldali, Bariş; Weißleder, Stephan; Engels, Gregor

doi:10.1016/b978-0-12-396535-6.00001-6

Cited by 6 publications

(12 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To date, the ROI of model‐based testing has not been sufficiently understood, and only some rough ROI calculations are available in the literature . This is reflected in little evidence about efficiency measures.…”

Section: Resultsmentioning

confidence: 99%

Model-based security testing: a taxonomy and systematic classification

Felderer

Zech

Breu

et al. 2015

Softw. Test. Verif. Reliab.

View full text Add to dashboard Cite

Model-based security testing relies on models to test whether a software system meets its security requirements. It is an active research field of high relevance for industrial applications, with many approaches and notable results published in recent years. This article provides a taxonomy for model-based security testing approaches. It comprises filter criteria (i.e. model of system security, security model of the environment and explicit test selection criteria) as well as evidence criteria (i.e. maturity of evaluated system, evidence measures and evidence level). The taxonomy is based on a comprehensive analysis of existing classification schemes for model-based testing and security testing. To demonstrate its adequacy, 119 publications on model-based security testing are systematically extracted from the five most relevant digital libraries by three researchers and classified according to the defined filter and evidence criteria. On the basis of the classified publications, the article provides an overview of the state of the art in model-based security testing and discusses promising research directions with regard to security properties, coverage criteria and the feasibility and return on investment of model-based security testing. 120 M. FELDERER ET AL. 9126 [4] defining security as a functional quality characteristic. However, it seems desirable that security testing directly targets the previous security properties, as opposed to taking the detour of functional tests of security mechanisms. This view is supported by the ISO/IEC 25010 [2] standard that revises ISO/IEC 9126 and introduces security as a new quality characteristic that is not included in the characteristic functionality any more.Because the former kind of (non-functional) security properties describes all executions of a system, this kind of security testing is intrinsically hard. Because testing cannot show the absence of faults, an immediately useful perspective directly considers the violation of these properties. This has resulted in the development of specific testing techniques such as penetration testing that simulates attacks to exploit vulnerabilities. Penetration tests are difficult to craft because tests often do not directly cause observable security exploits, and because the testers must think like an attacker [5], which requires specific expertise. During penetration testing, testers build a mental model of security properties, security mechanisms and possible attacks against the system and its environment. It seems intuitive that security testing can benefit from specifying these security test models in an explicit and processable way. Security test models provide guidance for the systematic and effective specification and documentation of security test objectives and security test cases, as well as for their automated generation and evaluation.The variant of testing that relies on explicit models that encode information on the system under test and/or its environment is called model-based testing (MBT) [6,7]. Especially in ...

show abstract

Section: Resultsmentioning

confidence: 99%

Model-based security testing: a taxonomy and systematic classification

Felderer

Zech

Breu

et al. 2015

Softw. Test. Verif. Reliab.

View full text Add to dashboard Cite

show abstract

“…In the experience of Section 3 of the overview article [16], there is evidence that as much as 40% of the effort is spent in modeling. In the experience of Section 3 of the overview article [16], there is evidence that as much as 40% of the effort is spent in modeling.…”

Section: Limitationsmentioning

confidence: 99%

“…Of course, the test suite must be based on the SCJ specification only and not be tailored for a particular implementation of the SCJ technology. The specification is written in natural language (English), where the detailed behavior of Java constructs such as classes and methods is documented in the Javadoc standard format.Writing test code in a conventional manner is laborious and often difficult, even when a framework such as JUnit [15] is used; also, it may be time-consuming to change the test code to match changes in the specification.Because the tests should check conformance with a specification, we decided to try a model-based approach [16]. As argued in previous works [13,14], a first step in this approach formalizes the informal specification in the JML [17,18].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Conformance test development with the Java modeling language

Søndergaard

Korsholm

Ravn

2017

Concurrency and Computation

View full text Add to dashboard Cite

In order to claim conformance with a Java Specification Request, a Java implementation has to pass all tests in an associated Technology Compatibility Kit. This paper presents a model-based development of a Technology Compatibility Kit test suite and a test execution tool for the draft safety-critical Java profile specification. The Java Modeling Language is used to model conformance constraints for the profile. Java Modeling Language annotations define contracts for classes and interfaces. The annotations are translated by a tool into runtime assertion checks. Hereby, the design and elaboration of the concrete test cases are simplified, because the expected results are derived from contracts and thus do not need to be provided explicitly. Bottom-up testing is applied for testing methods of the safety-critical Java classes, whereas topdown testing is applied for testing global properties, such as protocols, memory management, and real-time properties, including scheduling. The tests are executed using a simplified version of JUnit, which makes the test suite executable on resource-constrained platforms. Furthermore, Section 2 gives an overview of SCJ and its three conformance levels and concludes with a short overview of Java Modeling Language (JML).It is probably impossible to develop a test suite without familiarity with the language being tested. In our case, we have followed the development of the SCJ specification closely, commented on intermediate draft specifications and, over the years, acquired considerable insight in the details of the specification, as we have developed several prototype implementations [4][5][6]; the most recent one includes all three SCJ levels [7,8].This implementation is developed for resource-constrained embedded platforms, specifically the hardware-near virtual machine (HVM) [9]. Other prototype implementations are [6, 10,11].The draft specification work has nearly finished now (September 2016), although it has been delayed by a combined effort to revise the Real-Time Specification for Java (RTSJ) [12] to version 2.0. Because SCJ builds on RTSJ, this has led to many last minute changes to the SCJ specification. Yet, in order to complete the JSR process for SCJ, we began working on the TCK component in early 2013 [13] and reported on our progress in [14]. This paper consolidates the work of these papers and reports on the most recent progress with building the TCK. Furthermore, the approach is related to model-based testing in general, so it should be of interest for other developers of suites for conformance testing.A TCK is first of all a suite of tests checking whether an implementation of the SCJ technology conforms to the SCJ specification. Of course, the test suite must be based on the SCJ specification only and not be tailored for a particular implementation of the SCJ technology. The specification is written in natural language (English), where the detailed behavior of Java constructs such as classes and methods is documented in the Javadoc standard format.Writing test cod...

show abstract

“…These classical theories have their own representative models, including the finite state machine or the extended finite state machine, the UML models and the Markov chain. [1,2,3] Based on these classical theories and models,…”

Section: Introductionmentioning

confidence: 99%

An Embedded Software Testing Requirements Modeling Tool Describing Static and Dynamic Characteristics

Qu¹,

Cui²,

Zou³

et al. 2015

Proceedings of the 2015 International Symposium on Computers &Amp; Informatics

View full text Add to dashboard Cite

It is generally believed that Model-based testing (MBT) began in 1956. MBT is an important software quality assurance technology. We propose a set of graphical components describing the testing requirements of embedded software, and research how to plan test items and how to generate test cases. We couple some graphical components as a modeling system supporting to describe test requirement for embedded software. The modeling system proposed can enhance the efficiency of embedded software testing to a certain extent.

show abstract

Model-Based Testing: Achievements and Future Challenges

Cited by 6 publications

References 48 publications

Model-based security testing: a taxonomy and systematic classification

Model-based security testing: a taxonomy and systematic classification

Conformance test development with the Java modeling language

An Embedded Software Testing Requirements Modeling Tool Describing Static and Dynamic Characteristics

Contact Info

Product

Resources

About