The automation of business processes and decision-making has received major interest from practice and academia. As automation allows us to execute more processes (cases), monitoring automated decision-making is currently evolving into a big data analytics problem for companies. Thus, not only monitoring insights themselves, but also an effective use of such insights become important. In this context, the speed and ability to interpret data is closely related to the visualization of metrics and data. While various approaches for quantitative insights on automated decision-making have been proposed, there is currently no evidence as to how the specific visualization of such metrics helps companies to create more value from their data. In this report, we therefore present the results of an empirical experiment analyzing the cognitive effects of different visualization techniques for quantitative insights on understanding inconsistencies in automated decisionmaking data.