The chilling effects of algorithmic profiling: Mapping the issues

Büchi, Moritz; Fosch‐Villaronga, Eduard; Lutz, Christoph; Tamò-Larrieux, Aurelia; Velidi, Shruthi; Viljoen, Salomé

doi:10.1016/j.clsr.2019.105367

Cited by 41 publications

(41 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, the findings connect to the debate about algorithmic literacy. Particularly, knowledge about profiling and algorithmic literacy could be associated with chilling effects that affect those at the margins most strongly, potentially leading to a society with less space for nonconformity and alternative lifestyles (Büchi et al, 2019). In a follow-up study to the research presented here, we aim to connect perceptions of profiling to chilling effects.…”

Section: Resultsmentioning

confidence: 99%

“…Large Internet firms, such as Facebook and Google/Alphabet, as well as third-party data brokers collect and combine detailed personal data to create sophisticated profiles (Bach et al, 2019) used for predictive purposes. We understand profiling here as the systematic and purposeful recording and classification of data related to individuals (Büchi et al, 2019). Research has started to look into people's perception and engagement of algorithms (Bucher, 2017;Duffy et al, 2017;Eslami et al, 2015;Klawitter & Hargittai, 2018).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Perceptions of Algorithmic Profiling on Facebook and Their Social Implications

Larrieux

Fosch‐Villaronga

Velidi

et al. 2020

SPIR

Self Cite

View full text Add to dashboard Cite

With every digital interaction, individuals are increasingly subject to algorithmic profiling, understood as the systematic and purposeful recording and classification of data related to individuals. Large Internet firms, such as Facebook and Google/Alphabet, as well as third-party data brokers collect and combine detailed personal data to create sophisticated profiles for predictive purposes. Research has started to look into people’s perception and engagement of algorithms, showing that many users are unaware of the existence of algorithms, for example those which curate news feeds, and that a majority feels uncomfortable with algorithmic profiling on Facebook. In our research, we investigate perceptions of algorithmic profiling on Facebook by addressing the following questions: What user narratives of profiling on Facebook exist? What reactions do users have when confronted with Facebook’s inferred profiles? What are the social implications of user perceptions of profiling? Drawing on rich and recent survey data from 292 US-based Facebook users, we identified four overarching themes relating to Facebook's profiling activities: uncertainty, naiveté, realism, and fatalism. While the third theme is the most prevalent, Facebook is perceived as very powerful when it comes to algorithmic profiling. However, when confronted with their own profiles through the "My interests" and "My categories" sections in the Facebook Ad preferences menu, many users indicated surprise at how imprecise or even wrong some of the inferred interests and categories were. We discuss the social implications of our findings with regards social exclusion and social justice.

show abstract

Section: Resultsmentioning

confidence: 99%

mentioning

confidence: 99%

Perceptions of Algorithmic Profiling on Facebook and Their Social Implications

Larrieux

Fosch‐Villaronga

Velidi

et al. 2020

SPIR

Self Cite

View full text Add to dashboard Cite

show abstract

“…Already the anticipation of being identified in public accessible places can cause chilling effects on exercising fundamental rights [25] (pp. 125-126), [26].…”

Section: Biometric Ai Systems (Art 5(1)(d) Aia)mentioning

confidence: 99%

The European Commission’s Proposal for an Artificial Intelligence Act—A Critical Assessment by Members of the Robotics and AI Law Society (RAILS)

Ebers¹,

Hoch

Rosenkranz

et al. 2021

View full text Add to dashboard Cite

On 21 April 2021, the European Commission presented its long-awaited proposal for a Regulation “laying down harmonized rules on Artificial Intelligence”, the so-called “Artificial Intelligence Act” (AIA). This article takes a critical look at the proposed regulation. After an introduction (1), the paper analyzes the unclear preemptive effect of the AIA and EU competences (2), the scope of application (3), the prohibited uses of Artificial Intelligence (AI) (4), the provisions on high-risk AI systems (5), the obligations of providers and users (6), the requirements for AI systems with limited risks (7), the enforcement system (8), the relationship of the AIA with the existing legal framework (9), and the regulatory gaps (10). The last section draws some final conclusions (11).

show abstract

“…Legally, it is still unclear what remedies individuals will have against problematic ADM. Scholars have highlighted that discriminatory practices and potential chilling effects arising from the employment of such technologies are inadequately covered in the law, like data protection law (Wachter and Mittelstadt 2019 ; Büchi et al 2019 ). Accordingly, it becomes even more important that risks associated with data processing are identified, to allow users a more informed engagement.…”

Section: Transparency By Designmentioning

confidence: 99%

Towards Transparency by Design for Artificial Intelligence

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this article, we develop the concept of Transparency by Design that serves as practical guidance in helping promote the beneficial functions of transparency while mitigating its challenges in automated-decision making (ADM) environments. With the rise of artificial intelligence (AI) and the ability of AI systems to make automated and self-learned decisions, a call for transparency of how such systems reach decisions has echoed within academic and policy circles. The term transparency, however, relates to multiple concepts, fulfills many functions, and holds different promises that struggle to be realized in concrete applications. Indeed, the complexity of transparency for ADM shows tension between transparency as a normative ideal and its translation to practical application. To address this tension, we first conduct a review of transparency, analyzing its challenges and limitations concerning automated decision-making practices. We then look at the lessons learned from the development of Privacy by Design, as a basis for developing the Transparency by Design principles. Finally, we propose a set of nine principles to cover relevant contextual, technical, informational, and stakeholder-sensitive considerations. Transparency by Design is a model that helps organizations design transparent AI systems, by integrating these principles in a step-by-step manner and as an ex-ante value, not as an afterthought.

show abstract

The chilling effects of algorithmic profiling: Mapping the issues

Cited by 41 publications

References 57 publications

Perceptions of Algorithmic Profiling on Facebook and Their Social Implications

Perceptions of Algorithmic Profiling on Facebook and Their Social Implications

The European Commission’s Proposal for an Artificial Intelligence Act—A Critical Assessment by Members of the Robotics and AI Law Society (RAILS)

Towards Transparency by Design for Artificial Intelligence

Contact Info

Product

Resources

About