Agile validation of model transformations using compound F-Alloy specifications

Abstract

Model transformations play a key role in model driven software engineering approaches. Validation of model transformations is crucial for the quality assurance of software systems to be constructed. The relational logic based specification language Alloy and its accompanying tool the Alloy Analyzer have been used in the past to validate properties of model transformations. However Alloy based analysis of transformations suffers from several limitations. On one hand, it is time consuming and does not scale well. On the other hand, the reliance on Alloy, being a formal method, prevents the effective involvement of domain experts in the validation process which is crucial for pinpointing domain pertinent errors. Those limitations are even more severe when it comes to transformations whose input and/or output are themselves transformations (called compound transformations) because they are inherently more complex. To tackle the performance and scalability limitations, in previous work, we proposed an Alloy-based Domain Specific Language (DSL), called F-Alloy, that is tailored for model transformation specifications. Instead of pure analysis based validation, F-Alloy speeds up the validation of model transformations by applying a hybrid strategy that combines analysis with interpretation. In this paper, we formalize the notion of "hybrid analysis" and further extended it to also support efficient validation of compound transformations. To enable the effective involvement of domain experts in the validation process, we propose in this paper a new approach to model transformation validation, called Visualization-Based Validation (briefly VBV). Following VBV, representative instances of a to-bevalidated model transformation are automatically generated by hybrid analysis and shown to domain experts for feedback in a visual notation that they are familiar with. We prescribe a process to guide the application of VBV to model transformations and illustrate it with a benchmark model transformation.

Validation and verification in domain-specific modeling method engineering: an integrated life-cycle view

“…As the result, we achieve an integrated life-cycle of DSMM encompassing five phases, where V&V techniques are first class citizens. Please note that while doing that we also benefit from our earlier experiences, reported among others in [27,49,80], as well as experiences reported in the literature. Last but not least, building on the results of our analysis as well as on our subjective assessment of the state of the field of enterprise modeling, we formulate five desiderata, from the perspectives of practice, research and education, respectively, together with some initial ideas as to how they can be achieved.…”

“…On the semantic side, these example models basically constitute the extensional definition of the language. Omissions (e.g., missing well-formedness rules), excess (e.g., unnecessary concepts), or mismatching (e.g., a relation between wrong types of concepts) can be detected by simply reviewing these example models [48,49,112]. The reviewing activity is further enhanced when the example models are visualized in a domain-specific notation, because it makes possible for domain experts and non-technical users to understand the example models, hence, be able to pinpoint errors.…”

“…• Inter-operability testing * The choice of specific software verification technique to use depends highly on the meta-modeling platform used to engineer the DSMM, and the background and skill set of the language engineer validation of model transformations [49,89,90], and software inter-operability testing [128].…”

Model Transformation Testing and Debugging: A Survey

“…As described in the proposed conceptual model, and inline with the widely adopted notion of testing [198], we focus on testing approaches running the MT under test to identify failures, often referred to as łdynamic testingž [211]. Formal and static (i.e., those not running the MT under test) approaches for identifying bugs in MTs such as formal veriication [194,199,231,245,255,256] and model checking [262] have been largely studied in related papers and surveys [164,182,235] and are out of the scope of our work. We also excluded papers on model-based testing [208,213,218], where MTs are often used as a means to test other programs.…”

Ontologically correct taxonomies by construction

“…To support that the soundness results in Table 1 are significant, we rely on the small scope hypothesis, which basically claims most design errors can be found in small counterexamples [20]. Experimental results suggest that exhaustive testing within a small finite domain does indeed catch all type system errors in practice [21], and many case studies using the formal language and tool Alloy have confirmed the hypothesis by performing an analysis in a variety of scopes and showing, retrospectively, that a small scope would have sufficed to find all the bugs discovered [22].…”