Simulations in Recommender Systems: An industry perspective

Bernardi, Lucas; Batra, Sakshi; Bruscantini, Cintia A.

doi:10.48550/arxiv.2109.06723

Cited by 2 publications

(16 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The usage of simulated RL4Rec environments is widespread [5,24,29,46,48,49,61,63,68] and for a good reason: RL4Rec methods learn by directly interacting with users but the online nature of this learning process brings risks and limitations: (1) in practice, the user experience can be negatively affected during the early stages of the learning process; and (2) research and experimentation with RL4Rec systems is often infeasible since most researchers have no access to real interactions with live users. RS simulators mitigate these issues as they allow RS developers and researchers to optimize and evaluate their RL4Rec methods on simulated user behavior [5,24,46,48]. Some simulators generate user behavior based on fully synthetic data (e.g., generated from a Bernoulli distribution [46]).…”

Section: State Encodersmentioning

confidence: 99%

“…Some simulators generate user behavior based on fully synthetic data (e.g., generated from a Bernoulli distribution [46]). These have been critiqued for oversimplifying user behavior [5,48]. Alternatively, to match real user behavior more closely, other simulators generate user behavior based on logged user data [24,48,63].…”

Section: State Encodersmentioning

confidence: 99%

“…Recently, Huang et al [22] pointed out that simulators that do not debias logged user data yield RL4Rec methods that are heavily affected by selection bias. Hence, we use SOFA [22] the only publicly available debiased simulator [5] -to mitigate the effect of selection bias on the resulting RL4Rec methods. (2) Different RL method: DQN is the most popular RL method used in RL4Rec [9,10,25,33,34,36,44,53,60,65,67,68]; it is structurally simpler than the actor-critic method by only optimizing one objective.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

State Encoders in Reinforcement Learning for Recommendation: A Reproducibility Study

Huang,

Oosterhuis,

Cetinkaya

et al. 2022

Preprint

View full text Add to dashboard Cite

Methods for reinforcement learning for recommendation (RL4Rec) are increasingly receiving attention as they can quickly adapt to user feedback. A typical RL4Rec framework consists of (1) a state encoder to encode the state that stores the users' historical interactions, and (2) an RL method to take actions and observe rewards. Prior work compared four state encoders in an environment where user feedback is simulated based on real-world logged user data. An attention-based state encoder was found to be the optimal choice as it reached the highest performance. However, this finding is limited to the actor-critic method, four state encoders, and evaluationsimulators that do not debias logged user data. In response to these shortcomings, we reproduce and expand on the existing comparison of attention-based state encoders (1) in the publicly available debiased RL4Rec SOFA simulator with (2) a different RL method, (3) more state encoders, and (4) a different dataset. Importantly, our experimental results indicate that existing findings do not generalize to the debiased SOFA simulator generated from a different dataset and a Deep Q-Network (DQN)-based method when compared with more state encoders.

show abstract

Section: State Encodersmentioning

confidence: 99%

Section: State Encodersmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

State Encoders in Reinforcement Learning for Recommendation: A Reproducibility Study

Huang,

Oosterhuis,

Cetinkaya

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…One of the most promising approaches to cope with the above-mentioned issues is the usage of synthetic data and the modeling and simulation (M&S) of interactions between users and RSs Ekstrand et al (2021); Bernardi et al (2021); Kiyohara et al (2021); Balog et al (2022). It is worth mentioning here that the analysis of search requests in Google Scholar conducted in Ekstrand et al (2021) shows that among the papers published from 2017 to 2021 and presented at worldclass conferences on the subject of RSs, about 27% of papers use synthetic data and the M&S or discuss their applications to the task.…”

Section: Introductionmentioning

confidence: 99%

“…• to supplement and/or replace real-world data in the RS training and testing process with its synthetic analogues in the simulated environment and to overcome the data insufficiency problem del Carmen et al (2017); Ekstrand et al (2021) and the necessity to perform complex, costly and risky online experiments Kiyohara et al (2021); Bernardi et al (2021);…”

Section: Introductionmentioning

confidence: 99%

Synthetic Data-Based Simulators for Recommender Systems: A Survey

Stavinova¹,

Grigorievskiy²,

Volodkevich³

et al. 2022

Preprint

View full text Add to dashboard Cite

This survey aims at providing a comprehensive overview of the recent trends in the field of modeling and simulation (M&S) of interactions between users and recommender systems and applications of the M&S to the performance improvement of industrial recommender engines. We start with the motivation behind the development of frameworks implementing the simulations -simulators -and the usage of them for training and testing recommender systems of different types (including Reinforcement Learning ones). Furthermore, we provide a new consistent classification of existing simulators based on their functionality, approbation, and industrial effectiveness and moreover make a summary of the simulators found in the research literature. Besides other things, we discuss the building blocks of simulators: methods for synthetic data (user, item, user-item responses) generation, methods for what-if experimental analysis, methods and datasets used for simulation quality evaluation (including the methods that monitor and/or close possible simulation-to-reality gaps), and methods for summarization of experimental simulation results. Finally, this survey considers emerging topics and open problems in the field.

show abstract

Simulations in Recommender Systems: An industry perspective

Cited by 2 publications

References 0 publications

State Encoders in Reinforcement Learning for Recommendation: A Reproducibility Study

State Encoders in Reinforcement Learning for Recommendation: A Reproducibility Study

Synthetic Data-Based Simulators for Recommender Systems: A Survey

Contact Info

Product

Resources

About