Questioning the EU proposal for an Artificial Intelligence Act: The need for prohibitions and a stricter approach to biometric surveillance1

Abstract

Artificial Intelligence (AI)-based surveillance technologies such as facial recognition, emotion recognition and other biometric technologies have been rapidly introduced by both public and private entities all around the world, raising major concerns about their impact on fundamental rights, the rule of law and democracy. This article questions the efficiency of the European Commission’s Proposal for Regulation of Artificial Intelligence, known as the AI Act, in addressing the threats and risks to fundamental rights posed by AI biometric surveillance systems. It argues that in order to meaningfully address risks to fundamental rights the proposed classification of these systems should be reconsidered. Although the draft AI Act acknowledges that some AI practices should be prohibited, the multiple exceptions and loopholes should be closed, and in addition new prohibitions, in particular to emotional recognition and biometric categorisation systems, should be added to counter AI surveillance practices violating fundamental rights. The AI Act should also introduce stronger legal requirements, such as third-party conformity assessment, fundamental rights impact assessment, transparency obligations as well as enhance existing EU data protection law and the rights and remedies available to individuals, thus not missing the unique opportunity to adopt the first legal framework that truly promotes trustworthy AI.

Tough Decisions? Supporting System Classification According to the AI Act

“…Lim et al [4] argue that clear AI systems' classification is necessary to manage separate regulatory legislation. Barkane [12] suggests that the proposed classification should be reconsidered since there are multiple exceptions and loopholes. The inherent interrelatedness and the numerous possibilities for classification are problematic [13].…”

Algorithmic Social Sorting And New Legal Narratives On Digital Privacy

“…Individual privacy has been irreversibly exposed through the process of algorithmic (re)production of data as AI interferes in all forms of bureaucracy that are associated with the state, integrates into all aspects of labour, and even becomes a crucial part of entertainment, social and private life. Following the introduction of the GDPR, the EU is now introducing two new legal documents, namely, the Artificial Intelligence Act (AIA) 2 and the Digital Services Act (DSA) 3 which should also be applied internationally in the attempt to regulate the realm of the internet and AI on a global scale.…”

“…3 Digital Services Act, December 15, 2020, eur-lex.europa.eu/legal-content/EN/TXT/? uri= CELEX:52020PC0825 "…”