Algorithm Overdependence: How the Use of Algorithmic Recommendation Systems Can Increase Risks to Consumer Well-Being

Abstract

Consumers increasingly encounter recommender systems when making consumption decisions of all kinds. While numerous efforts have aimed to improve the quality of algorithm-generated recommendations, evidence has indicated that people often remain averse to superior algorithmic sources of information in favor of their own personal intuitions (a type II problem). The current work highlights an additional (type I) problem associated with the use of recommender systems: algorithm overdependence. Five experiments illustrate that, stemming from a belief that algorithms hold greater domain expertise, consumers surrender to algorithm-generated recommendations even when the recommendations are inferior. Counter to prior findings, this research indicates that consumers frequently depend too much on algorithm-generated recommendations, posing potential harms to their own well-being and leading them to play a role in propagating systemic biases that can influence other users. Given the rapidly expanding application of recommender systems across consumer domains, the authors believe that an appreciation and understanding of these risks is crucial to the effective guidance and development of recommendation systems that support consumer interests.

The Effect of Consumers’ Perceived Power and Risk in Digital Information Privacy: The Example of Cookie Notices

“…We thus contribute to the notice versus choice debate (e.g., Acquisti, Brandimarte, and Loewenstein 2015), suggesting that notice alone is insufficient. Finally, our finding of positive (negative) affective reactions due to increased power (risk) perceptions extends recent discussions about the effect of company and regulatory decisions on consumers' well-being (Banker and Khetani 2019;Komarova Loureiro et al 2016).…”

Optimizing the Future of Innovative Technologies and Infinite Data

“…Algorithms are now helping consumers sift through sizable and complex information to simplify and accelerate consumption decisions for many aspects of their lives. Banker and Khetani (2019) question whether algorithms always provide consumers with optimal decisions that are in their best interest. Through a series of five experiments, the authors show that consumers in certain consumption contexts and domains are overreliant on algorithm-generated recommendations, which can hurt them when the algorithms present suboptimal recommendations.…”

“…Noting that bias can exist in recommendation systems, Banker and Khetani (2019) suggest that GDPR principles be adopted to ensure rights of nondiscrimination and an informed explanation. These principles would pressure manufactures to reduce or nullify consumer exploitation.…”

“…In “ Geosurveillance, Location Privacy, and Personalization ,” Syagnik Banerjee (2019) examines how geopositioning-related data storage and flows can allow firms to legally surveil individuals, and emphasizes realities and risks associated with location privacy. In “ Algorithm Overdependence: How the Use of Algorithmic Recommendation Systems Can Increase the Risk to Consumer Well-Being ,” Sanchin Banker and Salil Khetani (2019) explore the use of algorithms within recommender systems and find that when consumers make decisions, they can overdepend on, and fall prey to, suboptimal or misinformed algorithm-infused recommendation systems.…”

“…Kamleitner and Mitchell (2019) point out that consumer data-privacy is determined by an interdependent network of individuals; the principles of Metcalf’s law indicate that as this network grows, so does the level of risk. As reliance on algorithms becomes frequent, the bias noted by Banker and Khetani (2019) can amplify risk. Indeed, other new technologies can track consumers’ location (Banerjee 2019), thus putting their information at risk and creating an opportunity for consumers to use it in a risky manner.…”