To VR or not to VR: Is virtual reality suitable to understand software development metrics?

Abstract

Background/Context: Currently, the usual interface for visualizing data is based on 2-D screens. Recently, devices capable of visualizing data while immersed in VR scenes are becoming common. However, it has not been studied in detail to which extent these devices are suitable for interacting with data visualizations in the specific case of data about software development.Objective/Aim: In this registered report, we propose to answer the following question: "Is comprehension of software development processes, via the visualization of their metrics, better when presented in VR scenes than in 2D screens?" In particular, we will study if answers obtained after interacting with visualizations presented as VR scenes are more or less correct than those obtained from traditional screens, and if it takes more or less time to produce those answers. Method: We will run an experiment with volunteer subjects from several backgrounds. We will have two setups: an on-screen application, and a VR scene. Both will be designed to be as much equivalent as possible in terms of the information they provide. For the former, we use a commercial-grade set of Kibana-based interactive dashboards that stakeholders currently use to get insights. For the latter, we use a set of visualizations similar to those in the on-screen case, prepared to provide the same set of data using the museum metaphor in a VR room. The field of analysis will be related to modern code review, in particular pull request activity. The subjects will try to answer some questions in both setups (some will work first in VR, some on-screen), which will be presented to them in random order. To draw results, we will compare and statistically analyze both the correctness of their answers, and the time spent until they are produced.