Creating personalized systems that people can scrutinize and control

Abstract

Widespread personalized computing systems play an already important and fast-growing role in diverse contexts, such as location-based services, recommenders, commercial Web-based services, and teaching systems. The personalization in these systems is driven by information about the user, a user model. Moreover, as computers become both ubiquitous and pervasive, personalization operates across the many devices and information stores that constitute the user's personal digital ecosystem. This enables personalization, and the user models driving it, to play an increasing role in people's everyday lives. This makes it critical to establish ways to address key problems of personalization related to privacy, invisibility of personalization, errors in user models, wasted user models, and the broad issue of enabling people to control their user models and associated personalization. We offer scrutable user models as a foundation for tackling these problems.This article argues the importance of scrutable user modeling and personalization, illustrating key elements in case studies from our work. We then identify the broad roles for scrutable user models. The article describes how to tackle the technical and interface challenges of designing and building scrutable user modeling systems, presenting design principles and showing how they were established over our twenty years of work on the Personis software framework. Our contributions are the set of principles for scrutable personalization linked to our experience from creating and evaluating frameworks and associated applications built upon them. These constitute a general approach to tackling problems of personalization by enabling users to scrutinize their user models as a basis for understanding and controlling personalization.

Providing awareness, explanation and control of personalized filtering in a social networking site

“…A survey of one movie recommender site shows that 86 % of surveyed users want an explanation feature added to the website (Herlocker et al 2000). The importance of awareness and explanation on improving user satisfaction (e.g., acceptance, trust) has been extensively discussed (Scheel et al 2014;Kay and Kummerfeld 2013;Sinha and Swearingen 2002). Herlocker et al (2000) mention some benefits provided by explaining recommendations such as: justification, user involvement, education and acceptance.…”

“…In recent years, a growing trend in personalized filtering research is to consider factors, other than accuracy, that contribute to the quality of filtering, because an overt focus on accuracy has limitations: it fails to consider human desires for variety and leads users to only receive information that matches their pre-stated preferences but fail to be exposed to new topics (referred to as the Bfilter bubble^problem) (Pariser 2011;Zhang et al 2012;Matt et al 2014;Loepp et al 2014). Moreover, current widespread personalized systems operate invisibly (Kay and Kummerfeld 2013). For personalized filtering, the platforms usually do not create awareness about what is being hidden from the user.…”

“…They are black boxes and provide no explanation about what data is filtered and why. The importance of awareness and explanation has been discussed in previous studies (Scheel et al 2014;Johnson and Johnson 1993), that lack of such transparency can lead to reduced user satisfaction (e.g., trust toward personalized filtering) in the system (Sinha and Swearingen 2002;Kay and Kummerfeld 2013;Lim et al 2009). Therefore, it is necessary to allow users to take control of the filtering process and its outcome (Dooms 2013;Loepp et al 2014).…”

Intelligent Computing in Personal Informatics

“…A key issue is enabling people to understand and control such processes, and creating a scrutable system can be an explicit design goal. Making system models scrutable can serve to increase engagement and trust, as well as to facilitate user-driven identification of inappropriate system behavior [24].…”

Managing information for personal goals (vision)

“…An over-riding concern in personalized systems (such as personal assistants) is "scrutability" [16], [17], so the user can understand how the system arrived at the information the user sees. For example, with data relevant to educational goals, the Asilomar convention for Learning research says "sharing practices must be made transparent to learners".…”