Visualization evaluation for cyber security

Abstract

The Visualization for Cyber Security research community (VizSec) addresses longstanding challenges in cyber security by adapting and evaluating information visualization techniques with application to the cyber security domain. This research effort has created many tools and techniques that could be applied to improve cyber security, yet the community has not yet established unified standards for evaluating these approaches to predict their operational validity. In this paper, we survey and categorize the evaluation metrics, components, and techniques that have been utilized in the past decade of VizSec research literature. We also discuss existing methodological gaps in evaluating visualization in cyber security, and suggest potential avenues for future research in order to help establish an agenda for advancing the state-of-the-art in evaluating cyber security visualizations.

EEVi - framework for evaluating the effectiveness of visualization in cyber-security

“…Staheli et al [20] presented a survey, in 2014, of 130 VizSec (IEEE Symposium on Visualization for Cyber Security) papers which showed that little research had been conducted in determining the effectiveness of cyber-security visualization. It also showed that 46% of these papers did not have any user-involvement in the evaluation phase.…”

“…Additionally, the evaluation techniques used to evaluate most tools were not effective in evaluating the visualizations that were produced based on the performed task nor were the evaluation techniques standardized. The lack of a common framework for standardized evaluation methods [20] has been concluded many times, but there is no research supporting the development of a framework to evaluate the effectiveness of cybersecurity visualization tools based on user requirements. Thus, creating a need for guidelines to standardize evaluation techniques and evaluate for effectiveness of performed task.…”

“…There is an outburst of cyber-security visualization tools, but the visualizations presented by these tools are rarely evaluated for effectiveness in terms of the task they aid in performing [20]. Furthermore, most of the visuals are developed and sometimes evaluated without any involvement of users [20].…”

“…Furthermore, most of the visuals are developed and sometimes evaluated without any involvement of users [20]. The lack of user-involvement could be a result of limited access to security analysts; or due to the nature of their jobs, security analysts cannot make significant time-commitment [16].…”

An Empirical Evaluation of the Effectiveness of Attack Graphs and Fault Trees in Cyber-Attack Perception

“…The visualisation of complex knowledge and information structures helps learners to better perceive complex concepts. Visual methods such as AMTs have considerable value in aiding cyber-attack perception [10,11]. Fithen et al, [12] outline the benefits of attack visualisation by arguing that AMTs enable non-experts to better understand and interpret attack models with little reference to logical models.…”