Measuring "Why" in Recommender Systems: a Comprehensive Survey on the Evaluation of Explainable Recommendation

Abstract

Explainable recommendation has shown its great advantages for improving recommendation persuasiveness, user satisfaction, system transparency and among others. A fundamental problem of explainable recommendation is how to evaluate the explanations. In the past few years, various evaluation strategies have been proposed. However, they are scattered in different papers, and there lacks a systematic and detailed comparison between them. To bridge this gap, in this paper, we comprehensively review the previous work, and provide different taxonomies according to the evaluation perspectives and evaluation methods. Beyond summarization, we also analyze the (dis)advantages of existing evaluation methods, and provide a series of guidelines on how to select them. The contents of this survey are concluded from more than 100 papers from top-tier conferences like IJCAI, AAAI, TheWebConf, Recsys, UMAP and IUI, and the complete comparisons are presented at https://shimo.im/sheets/VKrpYTcwVH6KXgdy/MO DOC/. With this survey, we finally aim to provide a clear and comprehensive review on the evaluation of explainable recommendation.