An automated functional testing approach for virtual reality applications

Abstract

Software testing is regarded as an important method for fault revealing. Despite this advantage, it has been poorly used within the scope of virtual reality (VR) applications because they are highly complex and have peculiar features. Most testing performed of this VR applications are usability, which is conducted manually and only at final of the development process. Although some works try to propose criteria for this domain, there are no approaches that automatize the generation of test data from requirements specification in the VR domain. This paper proposes an approach called virtual reality-requirements specification and testing (VR-ReST) to assist the requirements specification through a semiformal language and uses structural test criteria to generate test requirements and test data automatically for VR applications using scene graph concepts. The paper also examines the empirical results concerning the cost-effectiveness of the approach for three different VR applications through two experiments. Mutation testing was used to evaluate effectiveness. We found that the approach achieved a high mutation score outperforming random testing, by 20%, on average. Our results also demonstrate that the approach is promising since it assists in writing and validating the requirements, as well as in reducing the risks of requirement specification by adopting a semiformal language. KEYWORDS requirements engineering, software testing, virtual reality INTRODUCTIONFor several years, effort has been put into this research area in tasks such as development of techniques, testing criteria [1,2], and automation of the testing activity [3,4,5]. Some studies are dedicated to changing, adapting, and improving traditional techniques to accommodate peculiarities of specific software domains, such as WEB applications [6,7,8], database applications [9, 10], and GUI testing [11,12]. Our interest here lies in the domain of virtual reality (VR) applications, with their specific characteristics and high complexity.VR systems are three-dimensional (3D) virtual environments (VEs), as opposed to traditional two-dimensional environments considered over the years in computational applications. Features such as interaction, immersion, and the importance of using conventional and unconventional input/output devices make VR programs highly complex [13]. Such features also add difficulty to the process of software development, in particular, requirements specification and software testing activities.VR applications have a powerful interface that allows advanced human-computer interaction by using human senses. Such interfaces enable the user to immerse, navigate, and interact in a three-dimensional environment using multi-sensory devices [14]. VR is being increasingly used, and it is regarded as a technology that considerably contributed to several application areas such as medicine [15,16], archeology [17], entertainment [18,19], and education [20].VR applications, like other complex systems, may involve features such as parallelism and synch...