Experimental Comparison of the Comprehensibility of a UML-based Formal Specification versus a Textual One

Abstract

The primary objective of software specification is to promote understanding of the system properties between stakeholders. Specification comprehensibility is essential particularly during software validation and maintenance as it permits the understanding of the system properties more easily and quickly prior to the required tasks. Formal notation such as B increases a specification's precision and consistency. However, the notation is regarded as being difficult to comprehend due to its unfamiliar symbols and rules of interpretation. Semi-formal notation such as the Unified Modelling Language (UML) is perceived as more accessible but it cannot be verified systematically to ensure a specification's accuracy. Integrating the UML and B could perhaps produce an accurate and approachable specification. This paper presents an experimental comparison of the comprehensibility of a UML-based graphical formal specification versus a purely textual formal specification. The measurement focused on the efficiency in performing the comprehension tasks. The experiment employed a cross-over design and was conducted on forty-one third-year and masters students. The results show that the integration of semi-formal and formal notations expedites the subjects' comprehension tasks with accuracy even with limited hours of training.

Table‐based formal specification approaches for control engineers—empirical studies of usability

“…Carew et al [46] experiment for the acceptance level of formal and informal specification and training time for formal language. Razali et al [47] present the usability experiment result of the combination of semi-formal and formal language with regard to comprehensibility and preferences, and conclude that the combination is useful in promoting specification. Timing-diagram-based language implies usability of the conventional timing-diagram, and patternbased approaches (e.g.…”

Untitled

“…Rational Rose provides the environment for the UML-B model development while U2B is a tool that translates a UML-B model to a B model so that it can be verified by B tools, such as Atelier-B (ClearSy, n.d.) and B-Toolkit (B-Core, 2002). The comprehensibility of the notation used in a UML-B model has been assessed in previous work (Razali, Snook, Poppleton, Garratt, & Walters, 2007). The assessment was conducted as a controlled experiment that compared a UML-B model and a B model for model interpretation task.…”

Empirical Formal Methods: Guidelines for Performing Empirical Studies on Formal Methods

“…They "found no compelling quantitative evidence that formal design techniques alone produced code of higher quality than informal design techniques", yet "conclude that formal design, combined with other techniques, yielded highly reliable code". We are also aware of some well-conducted controlled experiments for the comprehensibility of FM like Petri nets [41], Z [42,43], OBJ [44,45], and B [46,47], for a set of state-based (semi-)formal languages like Statecharts and the Requirements State Machine Language (RSML) [48], and for domain-specific methods and languages in business process modelling [49][50][51], software product lines [52,53] and security [54][55][56].…”