Data Analytics and Skeptical Actions: The Countervailing Effects of False Positives and Consistent Rewards for Skepticism

Abstract

We investigate if varying rates of false positives impact auditor skepticism toward red flags identified by data analytic tools. We also examine the extent to which consistent rewards for skepticism can improve the application of skepticism on audits employing data analytics. Using an experiment with practicing auditors we observe that when false positive rates are higher, skepticism levels are low. We also find that consistent rewards for skepticism significantly improve the skepticism of our auditors. However, the positive effect of rewards is limited in that we do not see improvements in skepticism when the false positive rate is higher and additional investigation is less likely to identify a misstatement. Our findings highlight the importance of calibrating analytic tools to reduce false positives and the need for a culture change where appropriate skepticism is consistently rewarded in order for audit firms to effectively use analytic tools to enhance audit quality.

The effects of person‐specific, task, and environmental factors on digital transformation and innovation in auditing: A review of the literature

“…When primed with the audit execution intervention, the reviewers perceived the two audit approaches to be similar in quality. The results complement Cao et al (2021) and other research (e.g., Brown‐Liburd, Brown‐Liburd, et al, 2021; Brown‐Liburd, Brazel, et al, 2021) that suggests stakeholder views (e.g., peer reviewers, regulators) influence auditors' willingness to adopt technology.…”

Behavioral challenges to professional skepticism in auditors’ data analytics journey

“…False positives (i.e., type I errors) are those items or relationships identified as potential anomalies that, after further investigation, are determined to be reasonable and explained variations in the data (e.g., AICPA 2017; Johnson and Wiley 2019; Barr-Pulliam et al 2020). Along with the larger number of anomalies identified by ADA, the presence of a larger number of false positives brings additional challenges to the exercise of professional skepticism (e.g., Cao et al 2015;Earley 2015;Krahel and Titera 2015;Vasarhelyi et al 2015;Wang and Cuthbertson 2015;Yoon et al 2015;Alles and Gray 2016;AICPA 2015AICPA , 2017Richins et al 2017;Salijeni et al 2019;Barr-Pulliam et al 2020;Kipp et al 2020;Austin et al 2021;Krieger et al 2021). Higher false positive rates of ADA indicate lower calibration rates and, hence, investigating anomalies leads to potentially excessive costs (e.g., audit reporting delays, budget overages, but also strained client relations).…”

“…Higher false positive rates of ADA indicate lower calibration rates and, hence, investigating anomalies leads to potentially excessive costs (e.g., audit reporting delays, budget overages, but also strained client relations). Given the usually high budget constraints faced by auditors, higher false positive rates may reduce auditors' motivation to investigate the anomalies identified by ADA, therefore reducing their motivation for skeptical behavior (Barr-Pulliam et al 2020).…”

“…Since the presence of a larger number of false positives has become a concern in the application practice of ADA, it is critical to explore how to counter negative effects of false positives on the application of professional skepticism (e.g., Salijeni et al 2019;Barr-Pulliam et al 2020;Austin et al 2021). Indeed, research finds that consistently rewarding auditors for their professional skepticism helps to motivate auditors' exercise of professional skepticism but only when false positive rates are lower (Barr-Pulliam et al 2020). Therefore, it seems that improving the calibration rates (i.e., reducing the false positive rates) of ADA is the primary solution to this issue.…”

“…Besides potential benefits, research shows that ADA may also bring challenges to the appropriate application of professional skepticism (e.g., Appelbaum et al 2017;Rose et al 2017;Barr-Pulliam et al 2020;Holmstrom 2020;Austin et al 2021;Commerford et al 2021;Holt and Loraas 2021). Being alert to the potential behavioral challenges and exploring approaches of mitigating those challenges are critical for audit practice to fully utilize the benefits of ADA.…”