A Reinforcement Learning Framework to Discover Natural Flavor Molecules

Abstract

Flavor is the focal point in the flavor industry, which follows social tendencies and behaviors. The research and development of new flavoring agents and molecules are essential in this field. However, the development of natural flavors plays a critical role in modern society. Considering this, the present work proposes a novel framework based on scientific machine learning to undertake an emerging problem in flavor engineering and industry. It proposes a combining system composed of generative and reinforcement learning models. Therefore, this work brings an innovative methodology to design new flavor molecules. The molecules were evaluated regarding synthetic accessibility, the number of atoms, and the likeness to a natural or pseudo-natural product. This work brings as contributions the implementation of a web scraper code to sample a flavors database and the integration of two scientific machine learning techniques in a complex system as a framework. The implementation of the complex system instead of the generative model by itself obtained 10% more molecules within the optimal results. The designed molecules obtained as an output of the reinforcement learning model’s generation were assessed regarding their existence or not in the market and whether they are already used in the flavor industry or not. Thus, we corroborated the potentiality of the framework presented for the search of molecules to be used in the development of flavor-based products.

Transfer Learning Approach to Develop Natural Molecules with Specific Flavor Requirements

“…Even though it amasses the generative and reinforcement learning techniques, its implementation is a completely different task and changes the system of deep neural networks. In this case, the new system can obtain the specific flavors required, unlike the framework described in the work of Queiroz et al (2023). 15 It is important to notice that the techniques presented throughout this work are avant-garde in flavor engineering.…”

“…In this case, the new system can obtain the specific flavors required, unlike the framework described in the work of Queiroz et al (2023). 15 It is important to notice that the techniques presented throughout this work are avant-garde in flavor engineering. 2.1.…”

“…The methodology of generation and improvement of the newly designed molecules for the nine families of flavors chosen to be generated follows the same methodology described in the mentioned works of Queiroz et al (2022). , Anyhow, the output of the learning system is newly designed molecules with specific properties and desired flavor, as illustrated in Figure .…”

“…The next layer is the implementation of reinforcement learning in the generative model, presented in the work of Queiroz et al (2023), 15 Section 2 in Figure 1, aiming to generate more accurate molecules and follow defined criteria and standards. The reinforcement learning model training required the definition of a reward based on the synthetic accessibility score and the natural product-likeness score, actions to be taken, and consistent metrics to evaluate the choice of actions.…”

“…While it builds upon the foundations of generative and reinforcement learning techniques, its implementation diverges significantly from previous approaches, necessitating a reconfiguration of the deep neural network system. In contrast to the framework outlined by Queiroz et al (2023), which is limited to general flavorings, this novel system is capable of precisely obtaining the desired flavors needed for a particular product. This offers a distinct advantage in the realm of flavor engineering as it allows for the creation of more targeted and customized flavor profiles, catering to the ever-evolving preferences of consumers and the competitive nature of the industry.…”