Reinforcement learning for personalization: A systematic literature review

Abstract

The major application areas of reinforcement learning (RL) have traditionally been game playing and continuous control. In recent years, however, RL has been increasingly applied in systems that interact with humans. RL can personalize digital systems to make them more relevant to individual users. Challenges in personalization settings may be different from challenges found in traditional application areas of RL. An overview of work that uses RL for personalization, however, is lacking. In this work, we introduce a framework of personalization settings and use it in a systematic literature review. Besides setting, we review solutions and evaluation strategies. Results show that RL has been increasingly applied to personalization problems and realistic evaluations have become more prevalent. RL has become sufficiently robust to apply in contexts that involve humans and the field as a whole is growing. However, it seems not to be maturing: the ratios of studies that include a comparison or a realistic evaluation are not showing upward trends and the vast majority of algorithms are used only once. This review can be used to find related work across domains, provides insights into the state of the field and identifies opportunities for future work.

pH-RL: A Personalization Architecture to Bring Reinforcement Learning to Health Practice

“…Many practical limitations arise in traditional domains such as healthcare, making these benefits listed above fade away [4,5,9,10]. Such limitations are the inaccessibility to large samples of data, the unavailability of environments to train and evaluate algorithms in, the limitations on the data caused by privacy laws, and safety concerns (e.g., unsafe actions and exploration), explainability, and legal responsibility [4,10]. As a consequence, the applicability of DRL in many practical tasks remains limited.…”

“…Reinforcement Learning for Clinical Applications A literature review has shown that the number of applications of RL has been increasing [10]. Applications in healthcare range from treating patients with Sepsis at the Intensive Care Unit to sending personalized messages in e-Health mobile applications.…”

“…Reinforcement Learning for Personalization in e-Health Applications More and more real-world applications using RL for personalization in e-Health are found in literature [10]. Work by [9] focuses on developing RL policies coupled with clustering techniques for personalizing health interventions.…”

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

“…Healthcare. Recently DRL has gained traction for applications such as personalized healthcare treatments [236]. Liu et al [237] proposed the first DRL framework for estimating the optimal dynamic treatment regimes from observational medical data using DRL to estimate the long term value function.…”

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

“… Written as a teaching book and is more than a review article  Useful for those who wish to learn about deep learning, reinforcement learning, how they combine as deep-RL and their applications in the real world [45]  Reviewed the basic algorithms of deep reinforcement learning in terms of research method, deep learning model involving RL and their application in various fields (e.g., AlphaGo, robotics and natural language processing)  General review of all reinforcement applications [46]  Addressed the use of deep learning in region-based and region-free detection frameworks for object grasping [47]  Presented the application of deep learning methods to generalised robotic grasp detection [48]  Presented robotics-specific learning, reasoning and embodiment challenges in deep learning [49]  Focused on the extreme end of the spectrum: how robots can acquire the learning capability through only a handful of trials and a few minutes; referred to this challenge as "micro-data reinforcement learning" [32]  Reviewed deep reinforcement learning-based intelligent soft robotics, including various algorithms, and examples of their application in real-world scenarios [50]  Highlighted the recent reinforcement learning algorithms used in robot manipulation  Small range of topics due to the lack of papers pertaining to manipulation systems [51]  Presented an intensive study of vision-based robotic grasp detection methods [52]  Presented a survey of studies that used machine learning for manipulation  Formalised the robot manipulation learning problem [53]  Introduced a framework of personalisation settings and used it in a systematic literature review  Reviewed existing solutions and evaluated the corresponding strategies [54]  Identified and discussed different aspects that influence a robotics task requiring the nontrivial use of the concept of affordances Previous reviews have summarized different topics that are associated with robotics, as indicated in Table 1. The two most closely related topics to robotic manipulation were discussed in [50] and [52].…”