CLEF 2017 NewsREEL Overview: A Stream-Based Recommender Task for Evaluation and Education

Abstract

Abstract. News recommender systems provide users with access to news stories that they find interesting and relevant. As other online, stream-based recommender systems, they face particular challenges, including limited information on users' preferences and also rapidly fluctuating item collections. In addition, technical aspects, such as response time and scalability, must be considered. Both algorithmic and technical considerations shape working requirements for realworld recommender systems in businesses. NewsREEL represents a unique opportunity to evaluate recommendation algorithms and for students to experience realistic conditions and to enlarge their skill sets. The NewsREEL Challenge requires participants to conduct data-driven experiments in NewsREEL Replay as well as deploy their best models into NewsREEL Live's 'living lab'. This paper presents NewsREEL 2017 and also provides insights into the effectiveness of NewsREEL to support the goals of instructors teaching recommender systems to students. We discuss the experiences of NewsREEL participants as well as those of instructors teaching recommender systems to students, and in this way, we showcase NewsREEL's ability to support the education of future data scientists.

News Recommender System Considering Temporal Dynamics and News Taxonomy

“…There is limited work on NRS that addresses the temporal dynamics issues. CLEF NewsREEL [12] offered a campaign-style evaluation lab where the goal was to make news recommendation that must respond to a request within a given time frame (100ms). It emphasizes on the timeliness of the news recommendation.…”

Continuous Evaluation of Large-Scale Information Access Systems: A Case for Living Labs

“…• NewsREEL has been used by practitioners, teachers at universities, and researchers. A survey amongst participants (Lommatzsch et al, 2017) has revealed that one of the main motivations for them to participate was to acquire new skills that are currently in high demand in industry. At the same time, NewsREEL has also been successfully embedded in teaching since students experienced factors associated with working in industry (Hopfgartner et al, 2016).…”

“…Several authors (Lommatzsch et al (2016); Domann et al ( 2016); Beck et al ( 2017)) have used APACHE SPARK and APACHE MAHOUT. The combinations facilitate periodically building new micro-batches to update the models.…”

NewsImages

“…The proposed NewsImages dataset has several connections to the NewsREEL dataset [12] originally published in the CLEF NewsREEL challenge. The NewsREEL dataset provides snippets from several German news portals and discussion groups.…”