Institutionalised Distrust and Human Oversight of Artificial Intelligence: Toward a Democratic Design of AI Governance under the European Union AI Act

Laux, Johann

doi:10.2139/ssrn.4377481

Cited by 5 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the promises of AI-centered solutions for automation and efficiency, the irreplaceable human element, characterized by subtlety and subjective judgment, is essential in verifying information accuracy and contextual relevance (Ninaus & Sailer, 2022). This human oversight is crucial for in-depth analysis and as a means to improve and fine-tune AI systems through feedback, ensuring their continuous improvement (Laux, 2023). The theoretical framework of our study serves as a foundation for future research, emphasizing the multidimensional approach to content verification, combining technology, education, and human discernment.…”

Section: What Are Th E Salient Takeaways Of Th Is Study?mentioning

confidence: 99%

A System to Ensure Information Trustworthiness in Artificial Intelligence Enhanced Higher Education

Ali Khan,

Kauttonen,

Aunimo

et al. 2024

JITE:Research

View full text Add to dashboard Cite

Aim/Purpose: The purpose of this paper is to address the challenges posed by disinformation in an educational context. The paper aims to review existing information assessment techniques, highlight their limitations, and propose a conceptual design for a multimodal, explainable information assessment system for higher education. The ultimate goal is to provide a roadmap for researchers that meets current requirements of information assessment in education. Background: The background of this paper is rooted in the growing concern over disinformation, especially in higher education, where it can impact critical thinking and decision-making. The issue is exacerbated by the rise of AI-based analytics on social media and their use in educational settings. Existing information assessment techniques have limitations, requiring a more comprehensive AI-based approach that considers a wide range of data types and multiple dimensions of disinformation. Methodology: Our approach involves an extensive literature review of current methods for information assessment, along with their limitations. We then establish theoretical foundations and design concepts for EMIAS based on AI techniques and knowledge graph theory. Contribution: We introduce a comprehensive theoretical framework for an AI-based multimodal information assessment system specifically designed for the education sector. It not only provides a novel approach to assessing information credibility but also proposes the use of explainable AI and a three-pronged approach to information evaluation, addressing a critical gap in the current literature. This research also serves as a guide for educational institutions considering the deployment of advanced AI-based systems for information evaluation. Findings: We uncover a critical need for robust information assessment systems in higher education to tackle disinformation. We propose an AI-based EMIAS system designed to evaluate the trustworthiness and quality of content while providing explanatory justifications. We underscore the challenges of integrating this system into educational infrastructures and emphasize its potential benefits, such as improved teaching quality and fostering critical thinking. Recommendations for Practitioners: Implement the proposed EMIAS system to enhance the credibility of information in educational settings and foster critical thinking among students and teachers. Recommendation for Researchers: Explore domain-specific adaptations of EMIAS, research on user feedback mechanisms, and investigate seamless integration techniques within existing academic infrastructure. Impact on Society: This paper’s findings could strengthen academic integrity and foster a more informed society by improving the quality of information in education. Future Research: Further research should investigate the practical implementation, effectiveness, and adaptation of EMIAS across various educational contexts.

show abstract

Section: What Are Th E Salient Takeaways Of Th Is Study?mentioning

confidence: 99%

A System to Ensure Information Trustworthiness in Artificial Intelligence Enhanced Higher Education

Ali Khan,

Kauttonen,

Aunimo

et al. 2024

JITE:Research

View full text Add to dashboard Cite

show abstract

“…There are emerging risk management frameworks, mandatory laws, and standards including the European Union's AI Act (EU AI Act) 8 , the National Institute of Standards and Technology's AI Risk Management Framework (NIST AI RMF) 9 , etc., been discussed for different purposes such as AI risk management, information security management [16], etc. A review by Barraza de la Paz et al [17] presents an overview of emerging versions of the NIST Cyber Security Framework (CSF), ISO/IEC 27001:2022, and MAGERIT frameworks.…”

Section: Background and Related Workmentioning

confidence: 99%

AI Governance: A Systematic Literature Review

Batool,

Zowghi,

Bano

2024

Preprint

View full text Add to dashboard Cite

As artificial intelligence (AI) transforms a wide range of sectors and drives innova- tion, it also introduces different types of risks that should be identified, assessed, and mitigated. Various AI governance frameworks have been released recently by governments, organizations, and companies to mitigate risks associated with AI. However, it can be challenging for AI stakeholders to have a clear picture of the available AI governance frameworks, tools, or models and analyze the most suitable one for their AI system. To fill the gap, we present the litera- ture to answer key questions: WHO is accountable for AI systems’ governance, WHAT elements are being governed, WHEN governance occurs within the AI development life cycle, and HOW it is implemented through frameworks, tools, policies, or models. Adopting the systematic literature review (SLR) methodol- ogy, this study meticulously searched, selected, and analyzed 28 articles, offering a foundation for understanding different facets of AI governance. The analysis is further enhanced by categorizing artifacts of AI governance under team-level governance, organization-level governance, industry-level governance, national- level governance, and international-level governance. The findings of this study on existing AI governance solutions can assist research communities in proposing comprehensive AI governance practices.

show abstract

“…14 does not provide much information on what will make human oversight effective and meaningful. Note that comprehensive human oversight involves the supervision of the AI system during the entire lifecycle; it should also ensure mandatory transparency measures-none of which are mentioned in the AIA, leading to a considerable lack of clarity and certainty (Laux, 2023). 31 In addition, the AIA fails to clarify the notion of algorithmic bias and fairness.…”

Section: Other Points Of Criticismmentioning

confidence: 99%

“…The previous sections have analyzed, in particular, the AIA's 31 The key problems with human overseers include over-reliance (i.e., automation bias) or under-reliance (i.e., algorithm aversion) on algorithmic systems (i.e., humans often misjudge the accuracy of algorithmic predictions), lack of competence, expertise, and training (i.e., humans lacking the required skills to oversee AI systems effectively), and false incentives (e.g., financial or commercial self-interests and possible capture by industry interests). To address these challenges, the following design principles might help: 1. justification and legitimacy (i.e., proof of competence, training, and authority), 2. periodical mandates (i.e., a rotation system for auditors fosters impartiality and shields them from being captured by AI developers interests), 3. collective decisions (i.e., having a team of diverse overseers improves decision-making), 4. limited competence of institutions (i.e., checks and balances and separation of powers; e.g., second-degree overseers provide checks on first-degree overseers), 5. justiciability and accountability (i.e., establishing appeals procedures and liability claims against human overseers); and 6. transparency (i.e., explainable AI and algorithmic transparency [e.g., the design and operation of human oversight practices should be made public], and procedure and performance transparency) (Laux, 2023).…”

Section: Key Takeawaysmentioning

confidence: 99%

Mitigating the adverse effects of AI with the European Union's artificial intelligence act: Hype or hope?

Wörsdörfer

2023

Glob Bus Org Exc

View full text Add to dashboard Cite

In light of the rise of generative AI and recent debates about the socio‐political implications of large‐language models, chatbots, and the like, this paper analyzes the E.U.’s Artificial Intelligence Act (AIA), the world's first comprehensive attempt by a government body to address and mitigate the potentially negative impacts of AI technologies. The paper critically analyzes the AIA from a business and computer ethics point of view—a perspective currently lacking in the academic (e.g., GBOE‐related) literature. It evaluates, in particular, the AIA's strengths and weaknesses and proposes reform measures that could help to strengthen the AIA. Among the AIA's strengths are its legally binding character, extra‐territoriality, ability to address data quality and discrimination risks, and institutional innovations such as the AI Board and publicly accessible logs and database for AI systems. Among its main weaknesses are its lack of effective enforcement, oversight, and control, absence of procedural rights and remedy mechanisms, inadequate worker protection, institutional ambiguities, insufficient funding and staffing, and inadequate consideration of sustainability issues. Reform suggestions include establishing independent conformity assessment procedures, strengthening democratic accountability and judicial oversight, introducing redress and complaint mechanisms, ensuring the participation and inclusion of workers, guaranteeing political independence of the AI Board, providing enhanced funding and staffing of market surveillance authorities, and mandating “green AI.”

show abstract

Institutionalised Distrust and Human Oversight of Artificial Intelligence: Toward a Democratic Design of AI Governance under the European Union AI Act

Cited by 5 publications

References 26 publications

A System to Ensure Information Trustworthiness in Artificial Intelligence Enhanced Higher Education

A System to Ensure Information Trustworthiness in Artificial Intelligence Enhanced Higher Education

AI Governance: A Systematic Literature Review

Mitigating the adverse effects of AI with the European Union's artificial intelligence act: Hype or hope?

Contact Info

Product

Resources

About