Explaining Recommendations: Design and Evaluation

Tintarev, Nava; Masthoff, Judith

doi:10.1007/978-1-4899-7637-6_10

Cited by 203 publications

(171 citation statements)

References 58 publications

Supporting

Mentioning

168

Contrasting

Unclassified

Order By: Relevance

“…More specifically, we measured effectiveness as the change of a participant's response to "I am interested in watching this movie" before and after watching the movie trailer. The change should be small for effective explanations, which help users to accurately gauge their interest in recommendations before consumption [21]. As asking participants to watch a full movie is unrealistic in this experimental context, we approximate this action with watching a trailer.…”

Section: Efficiencymentioning

confidence: 99%

“…When designing recommendation explanations, there are several "styles" [21] of explanations to choose from: casebased, collaborative-based [11], content-based [22], conversational [19], demographic-based, and knowledge-based [24]. For example, Amazon's "Customers Who Bought This Item Also Bought ..." is collaborative-based, while Pandora's "Based on what you've told us so far, we're playing this track because it features ..." is content-based.…”

Section: Design Space and Related Workmentioning

confidence: 99%

“…We evaluated explanations according to established rubrics from past research [21]. We selected the four rubrics that are most appropriate for our natural language explanations as defined below.…”

Section: Efficiencymentioning

confidence: 99%

See 2 more Smart Citations

Crowd-Based Personalized Natural Language Explanations for Recommendations

Chang

Harper

Terveen

2016

Proceedings of the 10th ACM Conference on Recommender Systems

View full text Add to dashboard Cite

Explanations are important for users to make decisions on whether to take recommendations. However, algorithm generated explanations can be overly simplistic and unconvincing. We believe that humans can overcome these limitations. Inspired by how people explain word-of-mouth recommendations, we designed a process, combining crowdsourcing and computation, that generates personalized natural language explanations. We modeled key topical aspects of movies, asked crowdworkers to write explanations based on quotes from online movie reviews, and personalized the explanations presented to users based on their rating history. We evaluated the explanations by surveying 220 MovieLens users, finding that compared to personalized tagbased explanations, natural language explanations: 1) contain a more appropriate amount of information, 2) earn more trust from users, and 3) make users more satisfied. This paper contributes to the research literature by describing a scalable process for generating high quality and personalized natural language explanations, improving on state-of-the-art content-based explanations, and showing the feasibility and advantages of approaches that combine human wisdom with algorithmic processes.

show abstract

Section: Efficiencymentioning

confidence: 99%

Section: Design Space and Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Crowd-Based Personalized Natural Language Explanations for Recommendations

Chang

Harper

Terveen

2016

Proceedings of the 10th ACM Conference on Recommender Systems

View full text Add to dashboard Cite

show abstract

“…Mechanisms for improving inspectability and control have been introduced to different classes of intelligent systems from open learner models [6,8], to autonomous systems [7], decision support [3,14], and recommender systems [13,25]. These studies have found that inspectability and control can have a positive effect on user experience as well as improved mental models.…”

Section: Inspectability In Intelligent Systemsmentioning

confidence: 99%

What am I not Seeing? An Interactive Approach to Social Content Discovery in Microblogs

Kang

Tintarev

Höllerer

et al. 2016

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. In this paper, we focus on the informational and user experience benefits of user-driven topic exploration in microblog communities, such as Twitter, in an inspectable, controllable and personalized manner. To this end, we introduce "HopTopics" -a novel interactive tool for exploring content that is popular just beyond a user's typical information horizon in a microblog, as defined by the network of individuals that they are connected to. We present results of a user study (N=122) to evaluate HopTopics with varying complexity against a typical microblog feed in both personalized and non-personalized conditions. Results show that the HopTopics system, leveraging content from both the direct and extended network of a user, succeeds in giving users a better sense of control and transparency. Moreover, participants had a poor mental model for the degree of novel content discovered when presented with non-personalized data in the Inspectable interface.

show abstract

“…This includes considering factors such as diversity [43], [44], [45], novelty [42], [46], and serendipity [16], [47] alongside accuracy. Along with the expansion of the spectrum of evaluation metrics, the nature of interactions between users and recommender systems, and the influence that user interface and interaction style have on user behaviour and overall recommendation experience [48], [49], [50] have also been attracting more attention.…”

Section: Recommender Systemsmentioning

confidence: 99%

Does Reviewer Recommendation Help Developers?

Kovalenko

Tintarev

Pasynkov³

et al. 2020

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

Abstract-Selecting reviewers for code changes is a critical step for an efficient code review process. Recent studies propose automated reviewer recommendation algorithms to support developers in this task. However, the evaluation of recommendation algorithms, when done apart from their target systems and users (i.e., code review tools and change authors), leaves out important aspects: perception of recommendations, influence of recommendations on human choices, and their effect on user experience. This study is the first to evaluate a reviewer recommender in vivo. We compare historical reviewers and recommendations for over 21,000 code reviews performed with a deployed recommender in a company environment and set out to measure the influence of recommendations on users' choices, along with other performance metrics. Having found no evidence of influence, we turn to the users of the recommender. Through interviews and a survey we find that, though perceived as relevant, reviewer recommendations rarely provide additional value for the respondents. We confirm this finding with a larger study at another company. The confirmation of this finding brings up a case for more user-centric approaches to designing and evaluating the recommenders. Finally, we investigate information needs of developers during reviewer selection and discuss promising directions for the next generation of reviewer recommendation tools. Preprint: https://doi.org/10.5281/zenodo.1404814

show abstract

Explaining Recommendations: Design and Evaluation

Cited by 203 publications

References 58 publications

Crowd-Based Personalized Natural Language Explanations for Recommendations

Crowd-Based Personalized Natural Language Explanations for Recommendations

What am I not Seeing? An Interactive Approach to Social Content Discovery in Microblogs

Does Reviewer Recommendation Help Developers?

Contact Info

Product

Resources

About