JTorX: A Tool for On-Line Model-Driven Test Derivation and Execution

Belinfante, Axel

doi:10.1007/978-3-642-12002-2_21

Cited by 56 publications

(46 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…JTorX [7,3] was (partly) developed during the Quasimodo project [6]. It improves over its predecessor TorX [9,33]-which was one of the first modelbased testing tools in the field.…”

Section: Model-based Testingmentioning

confidence: 99%

Experiences with formal engineering: Model-based specification, implementation and testing of a software bus at Neopost

Sijtema

Belinfante

Stoelinga

et al. 2014

Science of Computer Programming

View full text Add to dashboard Cite

Abstract. We report on the actual industrial use of formal methods during the development of a software bus. At Neopost Inc., we developed the server component of a software bus, called the XBus, using formal methods during the design, validation and testing phase: We modeled our design of the XBus in the process algebra mCRL2, validated the design using the mCRL2-simulator, and fully automatically tested our implementation with the model-based test tool JTorX. This resulted in a well-tested software bus with a maintainable architecture. Writing the model, simulating it, and testing the implementation with JTorX only took 17% of the total development time. Moreover, the errors found with model-based testing would have been hard to find with conventional test methods. Thus, we show that formal engineering can be feasible, beneficial and cost-effective.

show abstract

“…JTorX [7,3] was (partly) developed during the Quasimodo project [6]. It improves over its predecessor TorX [9,33]-which was one of the first modelbased testing tools in the field.…”

Section: Model-based Testingmentioning

confidence: 99%

Experiences with formal engineering: Model-based specification, implementation and testing of a software bus at Neopost

Sijtema

Belinfante

Stoelinga

et al. 2014

Science of Computer Programming

View full text Add to dashboard Cite

show abstract

“…JTorX and TorXakis JTorX [14] is an update of the model-based testing tool TorX [54]. TorX is a model-based testing tool that uses labeled transition systems to derive and execute tests (execution traces) based on ioco [53] , a theory for defining when an implementation of a given specification is correct.…”

Section: Model Based Testingmentioning

confidence: 99%

“…Constructing informative test queries (i.e., membership queries) is known as test selection and is one of the main problems dealt with in model-based software testing [44]. Several software testing methods and tools have been developed for this task in order to aid software development and maintainance, see, e.g., [14,47,51]. Although, these methods are able to approximate equivalence queries by (smartly) asking many membership queries, an exponential amount of them (or one of exponential length) are required in the worst case.…”

Section: Introductionmentioning

confidence: 99%

Improving active Mealy machine learning for protocol conformance testing

et al. 2013

View full text Add to dashboard Cite

Using a well-known industrial case study from the verification literature, the bounded retransmission protocol, we show how active learning can be used to establish the correctness of protocol implementation I relative to a given reference implementation R. Using active learning, we learn a model M R of reference implementation R, which serves as input for a model based testing tool that checks conformance of implementation I to M R . In addition, we also explore an alternative approach in which we learn a model M I of implementation I, which is compared to model M R using an equivalence checker. Our work uses a unique combination of software tools for model construction (Uppaal), active learning (LearnLib, Tomte), model-based testing (JTorX, TorXakis) and verification (CADP, MRMC). We show how these tools can be used for learning models of and revealing errors in implementations, present the new notion of a conformance oracle, and demonstrate how conformance oracles can be used to speed up conformance checking.

show abstract

“…We used JTorX to test the implementation against mCRL2 model m dev . JTorX [13,14] is a model-based testing tool capable of automatic test generation, execution and evaluation. During the design phase, we already catered for model-based testing: we designed for testability by taking care that at the model boundaries, we could observe meaningful messages.…”

Section: First Phase: Developing the Xbusmentioning

confidence: 99%

Experiences with Formal Engineering: Model-Based Specification, Implementation and Testing of a Software Bus at Neopost.

Sijtema¹,

Stoelinga

Belinfante

et al. 2011

Formal Methods for Industrial Critical Systems

Self Cite

View full text Add to dashboard Cite

We report on the actual industrial use of formal methods during the development of a software bus. During an internship at Neopost Inc., of 14 weeks, we developed the server component of a software bus, called the XBus, using formal methods during the design, validation and testing phase: we modeled our design of the XBus in the process algebra mCRL2, validated the design using the mCRL2-simulator, and fully automatically tested our implementation with the model-based test tool JTorX. This resulted in a well-tested software bus with a maintainable architecture. Writing the model (m dev ), simulating it, and testing the implementation with JTorX only took 17% of the total development time. Moreover, the errors found with model-based testing would have been hard to find with conventional test methods. Thus, we show that formal engineering can be feasible, beneficial and cost-effective.The findings above, reported earlier in [1], were well-received, also in industrially oriented conferences [2,3]. In this paper, we look back on the case study, and carefully analyze its merits and shortcomings. We reflect on (1) the added benefits of model checking, (2) model completeness and (3) the quality and performance of the test process.Thus, in a second phase, after the internship, we model checked the XBus protocol-this was not done in [1] since the Neopost business process required a working implementation after 14 weeks. We used the CADP tool evaluator4 to check the behavioral requirements obtained during the development. Model checking did not uncover errors in model m dev , but revealed that model m dev was not complete and optimized: in particular, requirements to so-called bad weather behavior (exceptions, unexpected inputs, etc.) were missing. Therefore, we created several improved models, checked that we could validate them, and used them to analyze quality and performance of the test process. Model checking was expensive: it took us approx. 4 weeks in total, compared to 3 weeks for the entire model-based testing approach during the internship.In the second phase, we analyzed the quality and performance of the test process, where we looked at both code and model coverage. We found that high code coverage (almost 100%) is in most cases obtained within 1000 test steps and 2 minutes, which matches the fact that the faults in the XBus were discovered within a few minutes.Summarizing, we firmly believe that the formal engineering approach is costeffective, and produces high quality software products. Model checking does yield significantly better models, but is also costly. Thus, system developers should trade off higher model quality against higher costs.

show abstract

JTorX: A Tool for On-Line Model-Driven Test Derivation and Execution

Cited by 56 publications

References 8 publications

Experiences with formal engineering: Model-based specification, implementation and testing of a software bus at Neopost

Experiences with formal engineering: Model-based specification, implementation and testing of a software bus at Neopost

Improving active Mealy machine learning for protocol conformance testing

Experiences with Formal Engineering: Model-Based Specification, Implementation and Testing of a Software Bus at Neopost.

Contact Info

Product

Resources

About