Usability evaluation guidelines for business intelligence applications

Abstract

Business Intelligence (BI) applications provide business information and drive decision support. To derive maximum benefit from BI applications, the applications need to be used optimally. Optimal use depends on various factors including the usability of the product. The documented need for BI usability research together with the practical need for BI evaluation guidelines provides the rationale for this study. The purpose of the study was to investigate the usability evaluation of BI applications in the context of a coal mining organisation. In a mining environment companies need to derive maximum benefit from BI applications, therefore these applications need to be used optimally and that places the focus on usability. The research is guided by the question: What criteria should be used to evaluate the usability of BI applications. The research design included user observation, heuristic evaluation and a survey. Based on observations made during user support on a BI application used at a coal mining organisation a log of usability issues was compiled. The usability criteria extracted from this log was compared and contrasted with general usability criteria from literature to synthesize an initial set of BI usability evaluation criteria. These criteria were used as the basis for a heuristic evaluation of the BI application used at the coal mining organisation. The same BI application was also evaluated using the Software Usability Measurement Inventory (SUMI) standardised questionnaire. The results from the two evaluations were triangulated to provide a validated and refined set of criteria. The main contribution of the study is the usability evaluation criteria for BI applications presented as guidelines. These guidelines deviate from existing evaluation guidelines in the emphasis on information architecture, data quality and learnability.

Usability Evaluation for Business Intelligence Applications: A User Support Perspective

“…Interaction design and usability research reinforces the mantra that we should look at user's behaviour rather that listen to what they say and what they say they do [13]. This justifies the deeper analysis of the user log to refine the guidelines published previously as part of this study [14].…”

“…Based on a previous study [14] a set of BI usability evaluation criteria was presented. In this study that set of guidelines was re-evaluated in terms of a deeper analysis of the user identified BI usability issues, those insights were triangulated with the findings from the heuristic evaluating and the survey and interrogated in terms of more recent BI usability evaluation literature.…”

“…The BI guidelines proposed earlier in this study [14] were refined through a deeper analysis of the user issues as discussed in section 4.1, triangulation with the findings from the heuristic evaluating and the survey as well as a comparison with a recent BI usability evaluation study by Scholtz, Calitz & Snyman [50], to propose the new set of guidelines for the usability evaluation of BI applications as depicted in Table 3.…”

An Improved Usability Evaluation Model for Point-of-Sale Systems

“…The usability assessment for software applications has been widely investigated over the last few years [22][23][24][25][26][27][28][29][30]. Rajesh Kulkarni et al evaluated SUMI (Software Usability Measurement Inventory) model that assist users to understand the usability of inventory software [23].…”

“…Moreover, they gave guidelines for a new developer to design an application with a friendly user interface. Chrisna Jooste et al discussed usability assessment criteria and provided guidelines to improve usability of business applications after analyzing literature work, users' observation, heuristic evaluation and SUMI model [27]. Amir Bijarchian et al proposed a model to measure the usability of Enterprise Architecture (EA) framework [28].…”