Testing-based translation validation of generated code in the context of IEC 61508

Conrad, Mirko

doi:10.1007/s10703-009-0082-0

Cited by 21 publications

(10 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the previous work on code generator testing focuses on checking the correct functional behaviour of generated code .Most of these research efforts rely on the comparison of the model execution to the generated code execution. This is known in the software testing community as equivalence, comparative or back‐to‐back testing approach .…”

Section: Related Workmentioning

confidence: 99%

Leveraging metamorphic testing to automatically detect inconsistencies in code generator families

Boussaa

Barais

Sunyé

et al. 2019

Software Testing Verif & Rel

View full text Add to dashboard Cite

SUMMARY Generative software development has paved the way for the creation of multiple code generators that serve as a basis for automatically generating code to different software and hardware platforms. In this context, the software quality becomes highly correlated to the quality of code generators used during software development. Eventual failures may result in a loss of confidence for the developers, who will unlikely continue to use these generators. It is then crucial to verify the correct behaviour of code generators in order to preserve software quality and reliability. In this paper, we leverage the metamorphic testing approach to automatically detect inconsistencies in code generators via so‐called “metamorphic relations”. We define the metamorphic relation (i.e., test oracle) as a comparison between the variations of performance and resource usage of test suites running on different versions of generated code. We rely on statistical methods to find the threshold value from which an unexpected variation is detected. We evaluate our approach by testing a family of code generators with respect to resource usage and performance metrics for five different target software platforms. The experimental results show that our approach is able to detect, among 95 executed test suites, 11 performance and 15 memory usage inconsistencies.

show abstract

Section: Related Workmentioning

confidence: 99%

Leveraging metamorphic testing to automatically detect inconsistencies in code generator families

Boussaa

Barais

Sunyé

et al. 2019

Software Testing Verif & Rel

View full text Add to dashboard Cite

show abstract

“…As observed by Conrad, a translation validation approach, which is based on testing, seems to be a better solution in an engineering context. Therefore, we have concentrated our efforts in validating the transformation by testing.…”

Section: Validation Of the Transformationmentioning

confidence: 99%

“…Conrad proposes a translation validation workflow for the generated code in the context of the IEC 61508 standard. The translation validation process is comprised of (a) numeric equivalence testing between the generated code and the corresponding model, and (b) additional measures to demonstrate that unintended functionality has not been introduced during the translation process.…”

Section: Related Workmentioning

confidence: 99%

Design and validation of a C++ code generator from Abstract State Machines specifications

Bonfanti

Gargantini

Mashkoor

2019

J Software Evolu Process

View full text Add to dashboard Cite

According to best practices of model-driven engineering, the implementation of a system should be obtained from its model through a systematic model-to-code transformation. We present in this paper a methodology supported by the Asm2C++ tool, which allows the users to generate C++ code from abstract state machine models.Thanks to Asm2C++, the implementation is generated in a seamless manner with an assurance of potential bug freeness of the generated code. Following the same approach, model-based testing suggests deriving also (unit) tests from abstract models. We extend the Asm2C++ tool such that it can automatically produce unit tests for the generated code. Abstract test sequences, either generated randomly or through model checking, are translated to concrete C++ unit tests using the Boost library. In a similar manner, also, scenarios are generated in a behavior-driven development (BDD) approach. To guarantee the correctness of the transformation process, we define a mechanism to test the correctness of the model-to-code transformation with respect to two main criteria: syntactical correctness and semantic correctness, which is based on the definition of conformance between the specification and the code. Using this approach, we have devised a process able to test the generated code by reusing unit tests. The process has been used to validate our model-to-code transformations. KEYWORDSabstract state machine, automatic code generation, C++, model-driven engineering, transformation validation, unit tests generation | INTRODUCTIONFormal methods are often used to model requirements of a system under construction at the abstract level. Formal requirement models are then amenable to a series of quality assurance activities including dynamic analysis like simulation and static analysis like property verification. However, it may remain unclear how to properly link the final implementation to its abstract specification, since the implementation may contain platformcentric details, be written in a programming language, and running on a specific hardware. Performing a manual transformation of formal models into code increases costs, limits the reuse of a formal specification, is error prone as some faults can be introduced in the code writing process, and can be a barrier for a wider adoption of formal methods. It would be very useful to assist designers also in this very delicate last phase of the development process.Abstract state machine (ASM) 1 is a formal method that proposes a rigorous software and systems development process for seamless transformation of informal requirements into implementation. The ASM method is based on the design concept of refinement that allows to capture all

show abstract

“…To ensure the correctness of the controller implementation against the controller model, a typically used method in practice is equivalence testing (or back-to-back testing) [6,7,28] which compares the outputs of the executable model and code for the common input sequence. The limitation of this testingbased method is that it does not provide a thorough verification.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, for the case of non-equivalent model and code, LCV provides the LTI models obtained from the Simulink block diagram model and the C code respectively, so that the user can simulate both of the models and easily find an input sequence that leads to a discrepancy between their output behaviors. 6 Finally, for the case of equivalent model and code, LCV additionally provides a similarity transformation matrix T between the two LTI models, which is the key evidence to prove the input-output equivalence between the model and code.…”

Section: Verification Flow Of Linear Controller Verifiermentioning

confidence: 99%

LCV: A Verification Tool for Linear Controller Software

Park

Pajić

Sokolsky

et al. 2019

Tools and Algorithms for the Construction and Analysis of Systems

View full text Add to dashboard Cite

In the model-based development of controller software, the use of an unverified code generator/transformer may result in introducing unintended bugs in the controller implementation. To assure the correctness of the controller software in the absence of verified code generator/transformer, we develop Linear Controller Verifier (LCV), a tool to verify a linear controller implementation against its original linear controller model. LCV takes as input a Simulink block diagram model and a C code implementation, represents them as linear time-invariant system models respectively, and verifies an input-output equivalence between them. We demonstrate that LCV successfully detects a known bug of a widely used code generator and an unknown bug of a code transformer. We also demonstrate the scalability of LCV and a real-world case study with the controller of a quadrotor system.

show abstract

Testing-based translation validation of generated code in the context of IEC 61508

Cited by 21 publications

References 15 publications

Leveraging metamorphic testing to automatically detect inconsistencies in code generator families

Leveraging metamorphic testing to automatically detect inconsistencies in code generator families

Design and validation of a C++ code generator from Abstract State Machines specifications

LCV: A Verification Tool for Linear Controller Software

Contact Info

Product

Resources

About