Experimental evaluation of user performance on two-dimensional and three-dimensional perspective displays in discrete-event simulation

Abstract

Several experiments were carried out to compare the impacts of using a two dimensional (2D) plan view or a three dimensional (3D) perspective view in discrete event simulation visual displays. The experiments measured the performance of participants in spotting errors, describing the model, and suggesting improvements to the system. The participants using the 3D perspective display performed much better in spotting errors, taking on average about one third of the time of participants observing the 2D display. They also did much better in describing the model. There was no significant difference in suggesting improvements although this may have been because this task was easy. Most participants preferred the 3D perspective view when asked to compare the displays. The experiments indicate that the detailed design of the visual display may have a considerable effect on some of the tasks in a simulation project and hence on whether the overall project is successful.

Experimental behavioural research in operational research: What we know and what we might come to know

“…Studies such as Akpan & Brooks (2014) have modernised older work by Bell and his co-authors (e.g., Chau & Bell, 1995), showing that this finding holds for more modern higher quality and more detailed displays.…”

“…Bell & O'Keefe (1995), however, does demonstrate significant improvement by users over a solution they suggest prior to using the model. In a more recent study, Akpan & Brooks (2014) show that users of a 3D simulation animation achieve better understanding of a model than users of a 2D animation. In their study a between groups factorial design is used (so that different displays are made available to different subject groups) rather than measuring preference across all subjects.…”

“…Specifically, visualisations can increase the quality of decisions. Chau & Bell (1995) Akpan & Brooks (2014) …”

“…A number of studies have used students as subjects (e.g., O'Keefe & Pitt, 1991, Hämäläinen et al, 2013 or even colleagues (Akpan & Brooks, 2014). Better external validity is achieved with actual decision makers, or at least professionals.…”

“…There is no attempt to define the overall quality of the model (more an implicit assumption that good modellers will produce models above a threshold such that analysing model construction is a valid research activity Across all of these studies (those focused on model creation and decision making) there are few efforts to draw in antecedent variables. Hämäläinen et al (2013) and Akpan & Brooks (2014) collect data regarding age, gender, etc., but use it to assess the appropriateness of the subject group, rather than as part of the causal analysis.…”