They shall be fair, transparent, and robust: auditing learning analytics systems

Abstract

In the near future, systems, that use Artificial Intelligence (AI) methods, such as machine learning, are required to be certified or audited for fairness if used in ethically sensitive fields such as education. One example of those upcoming regulatory initiatives is the European Artificial Intelligence Act [AI Ethics 3:267-279, 2022]. Interconnected with fairness are the notions of system transparency (i.e. how understandable is the system) and system robustness (i.e. will similar inputs lead to similar results). Ensuring fairness, transparency, and robustness requires looking at data, models, system processes, and the use of systems as the ethical implications arise at the intersection between those. The potential societal consequences are domain specific, it is, therefore, necessary to discuss specifically for Learning Analytics (LA) what fairness, transparency, and robustness mean and how they can be certified. Approaches to certifying and auditing fairness in LA include assessing datasets, machine learning models, and the end-to-end LA process for fairness, transparency, and robustness. Based on Slade and Prinsloo’s six principals for ethical LA [(Artificial Intelligence Act) And Amending Certain Union Legislative Acts. AIA, 2021], relevant audit approaches will be deduced. Auditing AI applications in LA is a complex process that requires technical capabilities and needs to consider the perspectives of all stakeholders. This paper proposes a comprehensive framework for auditing AI applications in LA systems from the perspective of learners' autonomy, provides insights into different auditing methodologies, and emphasizes the importance of reflection and dialogue among providers, buyers, and users of these systems to ensure their ethical and responsible use.

Enhancing Fraud Detection in Financial Statements with Deep Learning: An Audit Perspective

“…Ethical Considerations and Regulatory Requirements Transparency in DL-Driven Auditing is a paramount ethical consideration (Simbeck, K, 2023). Auditors must ensure transparency in implementing DL algorithms, disclosing data sources, considered features, and decision-making processes (Simbeck, K, 2023). Transparent practices enable stakeholders to scrutinize DL model results, fostering trust and accountability (Waleed Hilal and al, (2022)).…”

“…This includes validating and verifying DL models and monitoring their impact on audit quality (Paul Munter ( 2022)). Establishing accountability mechanisms ensures auditors are held responsible for any adverse outcomes, promoting integrity in the profession (Simbeck, K, 2023). Regulatory Requirements for Auditors govern DL techniques in auditing, ensuring compliance with legal and ethical standards (Brown & Patel, 2021).…”

“…Regulatory Requirements for Auditors govern DL techniques in auditing, ensuring compliance with legal and ethical standards (Brown & Patel, 2021). Regulatory bodies like the SEC and PCAOB may set guidelines for AI and DL use in financial audits (Simbeck, K, 2023). These regulations cover data privacy, disclosure obligations, and audit documentation specific to DL-driven procedures (Johnson & Garcia, 2021).…”