Transforming Pharma with Data Science, AI and Machine Learning

Abstract

Continual RL is a challenging problem where the agent is exposed to a sequence of tasks; it should learn new tasks without forgetting old ones, and learning the new task should improve performance on previous and future tasks. The most common approaches use model-free RL algorithms as a base, and replay buffers have been used to overcome catastrophic forgetting. However, the buffers are often very large making scalability difficult. Also, the concept of replay comes from biological inspiration, where evidence suggests that replay is applied to a world model, which implies model-based RL -and model-based RL should have benefits for continual RL, where it is possible to exploit knowledge independent of the policy. We present WMAR, World Models with Augmented Replay, a model-based RL algorithm with a world model and memory efficient distribution matching replay buffer. It is based on the well-known DreamerV3 algorithm, which has a simple FIFO buffer and was not tested in a continual RL setting. We evaluated WMAR vs WMAR (FIFO only) on tasks with and without shared structure from OpenAI ProcGen and Atari respectively, and without a task oracle. We found that WMAR has favourable properties on continual RL with significantly reduced computational overhead compared to WMAR (FIFO only). WMAR had small benefits over DreamerV3 on tasks with shared structure and substantially better forgetting characteristics on tasks without shared structure; but at the cost of lower plasticity seen in a lower maximum on new tasks. The results suggest that model-based RL using a world model with a memory efficient replay buffer can be an effective and practical approach to continual RL, justifying future work.

Computational Pharmaceutics: A Comprehensive Review of Past Achievements, Present Applications, and Future Challenges

“…To fully embrace computational techniques in the pharmaceutical industry, significant cultural changes are essential. These changes include shifting mindsets towards accepting the predictive power of simulations in drug discovery (Hariry & Barenji, 2023), implementing robust data strategies that incorporate advanced analytics and machine learning algorithms, and adapting to digital transformations that encourage the use of new technologies like 3D printing and precision medicine (Yang, 2022). Additionally, the industry needs to prioritize innovation through research and development while ensuring compliance with evolving technological advancements (Edwards et al, 2021;Yang, 2022).…”

“…These changes include shifting mindsets towards accepting the predictive power of simulations in drug discovery (Hariry & Barenji, 2023), implementing robust data strategies that incorporate advanced analytics and machine learning algorithms, and adapting to digital transformations that encourage the use of new technologies like 3D printing and precision medicine (Yang, 2022). Additionally, the industry needs to prioritize innovation through research and development while ensuring compliance with evolving technological advancements (Edwards et al, 2021;Yang, 2022). Embracing a patient-centric approach and changing existing regulatory policies are also crucial cultural shifts required to leverage the benefits of artificial intelligence and big data in drug development (Maithani et al, 2022).…”

Computational Pharmaceutics: A Comprehensive Review of Past Achievements, Present Applications, and Future Challenges

“…To fully embrace computational techniques in the pharmaceutical industry, significant cultural changes are essential. These changes include shifting mindsets towards accepting the predictive power of simulations in drug discovery (Hariry & Barenji, 2023), implementing robust data strategies that incorporate advanced analytics and machine learning algorithms, and adapting to digital transformations that encourage the use of new technologies like 3D printing and precision medicine (Yang, 2022). Additionally, the industry needs to prioritize innovation through research and development while ensuring compliance with evolving technological advancements (Edwards et al, 2021;Yang, 2022).…”

“…These changes include shifting mindsets towards accepting the predictive power of simulations in drug discovery (Hariry & Barenji, 2023), implementing robust data strategies that incorporate advanced analytics and machine learning algorithms, and adapting to digital transformations that encourage the use of new technologies like 3D printing and precision medicine (Yang, 2022). Additionally, the industry needs to prioritize innovation through research and development while ensuring compliance with evolving technological advancements (Edwards et al, 2021;Yang, 2022). Embracing a patient-centric approach and changing existing regulatory policies are also crucial cultural shifts required to leverage the benefits of artificial intelligence and big data in drug development (Maithani et al, 2022).…”

Machine learning hypothesis-generation for patient stratification and target discovery in rare disease: our experience with Open Science in ALS

“…The convergence of artificial intelligence (AI), machine learning (ML), and data science is adding new dimensions to the advancement of our understanding of disease biology ( Yang, n.d. ). Traditional drug discovery and development is a high-risk, time- and cost-consuming process that takes, on average, over a decade and over $1 billion for each new drug approved for clinical use ( Schaduangrat et al, 2020 ; Sun et al, 2022 ).…”