Discovering diverse solutions in deep reinforcement learning by maximizing state–action-based mutual information

Osa, Takayuki; Tangkaratt, Voot; Sugiyama, Masashi

doi:10.1016/j.neunet.2022.04.009

Cited by 13 publications

(10 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Tales datos coinciden con los obtenidos por Según Deng et al (2022), los estilos de aprendizaje que ejecutan los infantes dependen de estímulos en el ámbito del aprendizaje, es decir, los contextos educativos bajo las cuales un estudiante es más factible que asimile. Para ello, Osa et al, (2022), iniciar conociendo a los estudiantes su forma de aprendizaje, sus métodos o formas como aprenden, la pasión por una asignatura, y en base a ello intentar personalizar la enseñanza en la medida de lo posible. Por su parte, Pupo & Torres (2010), señalan que, determinar los estilos de aprendizaje en los educandos puede proporcionar el acceso a un importante volumen de datos, y conocimientos para orientar en ellos de modo personal inculcando el desarrollo de la capacidad de creatividad para la resolución de problemas de cualquier índole social.…”

Section: Discussionunclassified

Estilos de aprendizaje y niveles de comprensión lectora en estudiantes de nivel primaria

Tafur Serván

2023

Pakamuros

View full text Add to dashboard Cite

El estudio tuvo como fin determinar la correlación entre los estilos de aprendizaje y los niveles de comprensión lectora en escolares del V ciclo de la I.E. N.º 18364, El Porvenir, Amazonas, 2021; la investigación fue de tipo descriptiva correlacional. El estudio contó con una población formado por 15 escolares. Para la recopilación de datos se manejó dos test; para la variable estilos aprendizaje, se empleó el test de Metts (1999) de acuerdo al modelo de Programación Neurolingüística-IEA VAK y para compresión lectora, se administró la prueba ACL–5, de Catala, con índices fiables de [0.701] y [0.947] consecuentemente. Las estadísticas evidenciaron que existe correlación significativa estadísticamente entre la variable estilos de aprendizaje y la comprensión de lectura (R=0.940; R2%= 89.25%; p<0.001).

show abstract

Section: Discussionunclassified

Estilos de aprendizaje y niveles de comprensión lectora en estudiantes de nivel primaria

Tafur Serván

2023

Pakamuros

View full text Add to dashboard Cite

show abstract

“…These two significant studies Xu [ 29 ] and Osa [ 30 ] collectively critiqued traditional ensemble learning methods and reinforcement learning algorithms for their lack of adaptability to noise, redundant features, and in learning diverse solutions. This underscores the necessity for more sophisticated, adaptable learning approaches.…”

Section: Literature Reviewmentioning

confidence: 99%

Advancing e-commerce user purchase prediction: Integration of time-series attention with event-based timestamp encoding and Graph Neural Network-Enhanced user profiling

Zhou,

Hudin

2024

PLoS ONE

View full text Add to dashboard Cite

In recent years, the global e-commerce landscape has witnessed rapid growth, with sales reaching a new peak in the past year and expected to rise further in the coming years. Amid this e-commerce boom, accurately predicting user purchase behavior has become crucial for commercial success. We introduce a novel framework integrating three innovative approaches to enhance the prediction model’s effectiveness. First, we integrate an event-based timestamp encoding within a time-series attention model, effectively capturing the dynamic and temporal aspects of user behavior. This aspect is often neglected in traditional user purchase prediction methods, leading to suboptimal accuracy. Second, we incorporate Graph Neural Networks (GNNs) to analyze user behavior. By modeling users and their actions as nodes and edges within a graph structure, we capture complex relationships and patterns in user behavior more effectively than current models, offering a nuanced and comprehensive analysis. Lastly, our framework transcends traditional learning strategies by implementing advanced meta-learning techniques. This enables the model to autonomously adjust learning parameters, including the learning rate, in response to new and evolving data environments, thereby significantly enhancing its adaptability and learning efficiency. Through extensive experiments on diverse real-world e-commerce datasets, our model demonstrates superior performance, particularly in accuracy and adaptability in large-scale data scenarios. This study not only overcomes the existing challenges in analyzing e-commerce user behavior but also sets a foundation for future exploration in this dynamic field. We believe our contributions provide significant insights and tools for e-commerce platforms to better understand and cater to their users, ultimately driving sales and improving user experiences.

show abstract

“…Achieving diversity has been studied in various contexts in RL (Mouret and Doncieux 2009;Mohamed and Rezende 2015;Eysenbach et al 2018;Osa, Tangkaratt, and Sugiyama 2021;Derek and Isola 2021). Eysenbach et al (Eysenbach et al 2018) proposed DIAYN to maximize the mutual information between states and skills, which results in a maximum entropy policy.…”

Section: Diversity In Reinforcement Learningmentioning

confidence: 99%

“…Eysenbach et al (Eysenbach et al 2018) proposed DIAYN to maximize the mutual information between states and skills, which results in a maximum entropy policy. Osa et al (Osa, Tangkaratt, and Sugiyama 2021) proposed a method that can learn infinitely The graphical model of diverse MDPs. Grey nodes are observed, and white nodes are hidden.…”

Section: Diversity In Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimization

Chen,

Huang,

Chiang

et al. 2024

AAAI

View full text Add to dashboard Cite

Most reinforcement learning algorithms seek a single optimal strategy that solves a given task. However, it can often be valuable to learn a diverse set of solutions, for instance, to make an agent's interaction with users more engaging, or improve the robustness of a policy to an unexpected perturbance. We propose Diversity-Guided Policy Optimization (DGPO), an on-policy algorithm that discovers multiple strategies for solving a given task. Unlike prior work, it achieves this with a shared policy network trained over a single run. Specifically, we design an intrinsic reward based on an information-theoretic diversity objective. Our final objective alternately constraints on the diversity of the strategies and on the extrinsic reward. We solve the constrained optimization problem by casting it as a probabilistic inference task and use policy iteration to maximize the derived lower bound. Experimental results show that our method efficiently discovers diverse strategies in a wide variety of reinforcement learning tasks. Compared to baseline methods, DGPO achieves comparable rewards, while discovering more diverse strategies, and often with better sample efficiency.

show abstract

Discovering diverse solutions in deep reinforcement learning by maximizing state–action-based mutual information

Cited by 13 publications

References 23 publications

Estilos de aprendizaje y niveles de comprensión lectora en estudiantes de nivel primaria

Estilos de aprendizaje y niveles de comprensión lectora en estudiantes de nivel primaria

Advancing e-commerce user purchase prediction: Integration of time-series attention with event-based timestamp encoding and Graph Neural Network-Enhanced user profiling

DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimization

Contact Info

Product

Resources

About