A Survey of Collaborative Reinforcement Learning: Interactive Methods and Design Patterns

Li, Zhaoxing; Shi, Lei; Cristea, Alexandra I.; Zhou, Yinsheng

doi:10.1145/3461778.3462135

Cited by 9 publications

(4 citation statements)

References 59 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, the developments in recommendation systems powered by deep learning have opened the door to investigating cutting-edge ideas, like reinforcement-learning-based recommendation systems. Although deep learning is excellent at identifying patterns and preferences in user interactions and content data, reinforcement learning goes one step further by integrating dynamic decision-making processes that modify suggestions over time in response to user input and interactions [26]. The notion of deep-learning-based recommendation systems is expanded upon as the concept of reinforcement learning is introduced.…”

Section: Deep-learning-based Recommendation Systemsmentioning

confidence: 99%

Integration of Deep Reinforcement Learning with Collaborative Filtering for Movie Recommendation Systems

Peng,

Siet,

Ilkhomjon

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

In the era of big data, effective recommendation systems are essential for providing users with personalized content and reducing search time on online platforms. Traditional collaborative filtering (CF) methods face challenges like data sparsity and the new-user or cold-start issue, primarily due to their reliance on limited user–item interactions. This paper proposes an innovative movie recommendation system that integrates deep reinforcement learning (DRL) with CF, employing the actor–critic method and the Deep Deterministic Policy Gradient (DDPG) algorithm. This integration enhances the system’s ability to navigate the recommendation space effectively, especially for new users with less interaction data. The system utilizes DRL for making initial recommendations to new users and to generate optimal recommendation as more data becomes available. Additionally, singular value decomposition (SVD) is used for matrix factorization in CF, improving the extraction of detailed embeddings that capture the latent features of users and movies. This approach significantly increases recommendation precision and personalization. Our model’s performance is evaluated using the MovieLens dataset with metrics like Precision, Recall, and F1 Score and demonstrates its effectiveness compared with existing recommendation benchmarks, particularly in addressing sparsity and new-user challenges. Several benchmarks of existing recommendation models are selected for the purpose of model comparison.

show abstract

Section: Deep-learning-based Recommendation Systemsmentioning

confidence: 99%

Integration of Deep Reinforcement Learning with Collaborative Filtering for Movie Recommendation Systems

Peng,

Siet,

Ilkhomjon

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…For example, a teacher can gain valuable information about their students, by observing their learning process and interactions and then design the most suitable and beneficial learning strategy for them [7]. However, the lack of teacher-student interactions in online learning environments makes the personalisation process extremely difficult [1,12]. In such online scenarios, student modelling can and has been applied, as a powerful tool to combat this issue [5].…”

Section: Student Modellingmentioning

confidence: 99%

Towards Student Behaviour Simulation: A Decision Transformer Based Approach

Shi

Zhou

et al. 2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

With the rapid development of Artificial Intelligence (AI), an increasing number of Machine Learning (ML) technologies have been widely applied in many aspects of life. In the field of education, Intelligence Tutoring Systems (ITS) have also made significant advancements using these technologies. Developing different teaching strategies automatically, according to mined student characteristics and learning styles, could significantly enhance students' learning efficiency and performance. This requires the ITS to recommend different learning strategies and trajectories for different individual students. However, one of the greatest challenges is the scarcity of data sets providing interactions between students and ITS, for training such ITS. One promising solution to this challenge is to train "sim students", which imitate real students' behaviour while using the ITS. The simulated interactions between these sim students and the ITS can then be generated and used to train the ITS to provide personalised learning strategies and trajectories to real students. In this paper, we thus propose SimStu, built upon a Decision Transformer, to generate learning behavioural data to improve the performance of the trained ITS models. The experimental results suggest that our SimStu could model real students well in terms of action frequency distribution. Moreover, we evaluate SimStu in an emerging ITS technology, Knowledge Tracing. The results indicate that SimStu could improve the efficiency of ITS training.

show abstract

“…In an MDP model, the agent observes the present state of the environment and takes actions on the environment in accordance with the policy, thereby changing the state of the environment and getting rewards. The ultimate goal of the agent is to reach the maximum cumulative reward, which is achieved using a reward function [16]. Figure 1 shows the structure of the MDP.…”

Section: Markov Decision Process (Mdp)mentioning

confidence: 99%

Sim-GAIL: A generative adversarial imitation learning approach of student modelling for intelligent tutoring systems

Li,

Shi,

Wang

et al. 2023

Neural Comput & Applic

Self Cite

View full text Add to dashboard Cite

The continuous application of artificial intelligence (AI) technologies in online education has led to significant progress, especially in the field of Intelligent Tutoring Systems (ITS), online courses and learning management systems (LMS). An important research direction of the field is to provide students with customised learning trajectories via student modelling. Previous studies have shown that customisation of learning trajectories could effectively improve students’ learning experiences and outcomes. However, training an ITS that can customise students’ learning trajectories suffers from cold-start, time-consumption, human labour-intensity, and cost problems. One feasible approach is to simulate real students’ behaviour trajectories through algorithms, to generate data that could be used to train the ITS. Nonetheless, implementing high-accuracy student modelling methods that effectively address these issues remains an ongoing challenge. Traditional simulation methods, in particular, encounter difficulties in ensuring the quality and diversity of the generated data, thereby limiting their capacity to provide intelligent tutoring systems (ITS) with high-fidelity and diverse training data. We thus propose Sim-GAIL, a novel student modelling method based on generative adversarial imitation learning (GAIL). To the best of our knowledge, it is the first method using GAIL to address the challenge of lacking training data, resulting from the issues mentioned above. We analyse and compare the performance of Sim-GAIL with two traditional Reinforcement Learning-based and Imitation Learning-based methods using action distribution evaluation, cumulative reward evaluation, and offline-policy evaluation. The experiments demonstrate that our method outperforms traditional ones on most metrics. Moreover, we apply our method to a domain plagued by the cold-start problem, knowledge tracing (KT), and the results show that our novel method could effectively improve the KT model’s prediction accuracy in a cold-start scenario.

show abstract

A Survey of Collaborative Reinforcement Learning: Interactive Methods and Design Patterns

Cited by 9 publications

References 59 publications

Integration of Deep Reinforcement Learning with Collaborative Filtering for Movie Recommendation Systems

Integration of Deep Reinforcement Learning with Collaborative Filtering for Movie Recommendation Systems

Towards Student Behaviour Simulation: A Decision Transformer Based Approach

Sim-GAIL: A generative adversarial imitation learning approach of student modelling for intelligent tutoring systems

Contact Info

Product

Resources

About