Deep neural network marketplace recommenders in online experiments

Eide, Simen; Zhou, Neng-Fa

doi:10.1145/3240323.3240387

Cited by 11 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, we need to consider the impact on the user experience if fairness affects relevance and satisfaction [26]. In a real scenario, the best approach would be to use contextual MABs based on deep learning [34] and/or meta-bandits [35], but for that we need to understand first the trade-offs involved. Future work includes using additional (larger) data sets with even more dynamic scenarios including adding new users and new items, as well as other distributions that may improve the balance between the knowledge of the real world and revenue.…”

Section: Discussionmentioning

confidence: 99%

Exploration Trade-offs in Web Recommender Systems

Baeza-Yates,

Delnevo

2022

2022 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

One of the main problems of web recommender systems is exposure bias, due to the fact that the web system itself is partly generating its own future, as users can only click on items shown to them. This bias not only creates popularity bias for products but also is one of the main challenges for recommender systems that deal with a very dynamic environment, where new items and users appear frequently and also user preferences change (or the market changes as happened with the coronavirus pandemic). The main paradigm to deal with these changes is to explore and exploit, avoiding the filter bubble effect. However, too much exploration also reduces short-term revenue and hence is usually traffic bounded. In this work, we present a counterfactual analysis that shows that web recommender systems could improve their long-term revenue if significantly more exploration is performed. This is good for the web recommender system but also for everyone as it creates more fair and healthy digital markets. This also improves the web user experience so is a double win-win for the e-commerce platform, the sellers, the users, and ultimately society.

show abstract

Section: Discussionmentioning

confidence: 99%

Exploration Trade-offs in Web Recommender Systems

Baeza-Yates,

Delnevo

2022

2022 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

show abstract

“…A possible enhancement of this framework, which we do not consider in this article, is to allow the item vectors to depend on other attributes of the items, such as image or text (e.g. Eide and Zhou (2018)).…”

Section: Hierarchical Item Priormentioning

confidence: 99%

Dynamic Slate Recommendation with Gated Recurrent Units and Thompson Sampling

Eide

Leslie

Frigessi

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We consider the problem of recommending relevant content to users of an internet platform in the form of lists of items, called slates. We introduce a variational Bayesian Recurrent Neural Net recommender system that acts on time series of interactions between the internet platform and the user, and which scales to real world industrial situations. The recommender system is tested both online on real users, and on an offline dataset collected from a Norwegian web-based marketplace, FINN.no, that is made public for research. This is one of the first publicly available datasets which includes all the slates that are presented to users as well as which items (if any) in the slates were clicked on. Such a data set allows us to move beyond the common assumption that implicitly assumes that users are considering all possible items at each interaction. Instead we build our likelihood using the items that are actually in the slate, and evaluate the strengths and weaknesses of both approaches theoretically and in experiments. We also introduce a hierarchical prior for the item parameters based on group memberships. Both item parameters and user preferences are learned probabilistically. Furthermore, we combine our model with bandit strategies to ensure learning, and introduce `in-slate Thompson Sampling' which makes use of the slates to maximise explorative opportunities. We show experimentally that explorative recommender strategies perform on par or above their greedy counterparts. Even without making use of exploration to learn more effectively, click rates increase simply because of improved diversity in the recommended slates.

show abstract

“…(5) ML pipeline incorporates the candidates, corresponding features, and click information to train a new model which can replace the existing production model in a one-off manner. Quick Access utilizes a deep network implementation for the ranking model, similar to other production recommendation systems [10,14]. The initial model was trained using TensorFlow [1] and utilized a pointwise learning-to-rank approach [23]-where for every user session, given the candidates that were generated to be ranked, the first user-clicked file is given a positive label and the other − 1 unclicked files given negative labels, no matter if the clicked file was clicked via the Quick Access UI or some other location within Drive.…”

Section: Initial System Overviewmentioning

confidence: 99%

“…This work also uses learning-to-rank to boost the quality of the system. [14] discusses deep learning recommenders at the popular Norwegian website FINN.no, where they use multi-armed bandits as a high-level reranker on top of other recommendations. For search, [16] gives an in-depth look at how deep learning enabled them to significantly improve Airbnb search.…”

Section: Related Workmentioning

confidence: 99%

Improving Recommendation Quality in Google Drive

Chen

Qin

Wilson

et al. 2020

Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery &Amp; Data Mining

View full text Add to dashboard Cite

Quick Access is a machine-learned system in Google Drive that predicts which files a user wants to open. Adding Quick Access recommendations to the Drive homepage cut the amount of time that users spend locating their files in half. Aggregated over thẽ 1 billion users of Drive, the time saved up adds up to~1000 work weeks every day. In this paper, we discuss both the challenges of iteratively improving the quality of a personal recommendation system as well as the variety of approaches that we took in order to improve this feature. We explored different deep network architectures, novel modeling techniques, additional data sources, and the effects of latency and biases in the UX. We share both pitfalls as well as successes in our attempts to improve this product, and also discuss how we scaled and managed the complexity of the system. We believe that these insights will be especially useful to those who are working with private corpora as well as those who are building a large-scale production recommendation system. CCS CONCEPTS • Information systems → Recommender systems; • Computing methodologies → Neural networks.

show abstract

Deep neural network marketplace recommenders in online experiments

Cited by 11 publications

References 15 publications

Exploration Trade-offs in Web Recommender Systems

Exploration Trade-offs in Web Recommender Systems

Dynamic Slate Recommendation with Gated Recurrent Units and Thompson Sampling

Improving Recommendation Quality in Google Drive

Contact Info

Product

Resources

About