Solvent extraction process design using deep reinforcement learning

Abstract

Many chemical manufacturing and separations processes like solvent extraction comprise hierarchically complex configurations of functional process units. With increasing complexity, strategies that rely on heuristics become less reliable for design optimization. In this study, we explore deep reinforcement learning for mapping the space of feasible designs to find an optimization strategy that can match or exceed the performance of conventional optimization. To this end, we implement a highly configurable learning environment for the solvent design process to which we can couple state‐of‐the‐art deep reinforcement learning agents. We evaluate the trained agents against the heuristic optimization for the solvent process design tasked to optimize recovery efficiency and product purity. Results demonstrated the agent successfully learned the strategy for predicting comparably optimal solvent extraction process designs for varying combinations of feed compositions.

Flowsheet generation through hierarchical reinforcement learning and graph neural networks

“…Recently, important first steps have been made toward using RL to synthesize novel process flowsheets 34‐39 . Midgley 34 introduced the “Distillation Gym”, an environment in which distillation trains for non‐azeotropic mixtures are generated by a soft‐actor‐critic RL agent and simulated in the open‐source process simulator COCO.…”

“…Then, a RL agents selects unit operations as discrete decisions using the economic value of the resulting process as objective. Furthermore, Plathottam et al 39 introduced a RL agent that optimizes a solvent extraction process by selecting discrete and continuous design variables within predefined flowsheets.…”

“…Recently, important first steps have been made toward using RL to synthesize novel process flowsheets. [34][35][36][37][38][39] Midgley 34 introduced the "Distillation Gym", an environment in which distillation trains for nonazeotropic mixtures are generated by a soft-actor-critic RL agent and simulated in the open-source process simulator COCO. The agent first decides whether to add a new distillation column to the intermediate flowsheet and subsequently selects continuous operating conditions.…”

“…Another gap in the literature concerns the combination of multiple unit operation types, recycle streams and a larger, hybrid action space. While previous works proposed these promising techniques in individual contributions, [34][35][36][37][38][39][40] they have not yet been combined to a unified framework.…”

Flowsheet synthesis through hierarchical reinforcement learning and graph neural networks

“…Recently, important first steps have been made towards using RL to synthesize novel process flowsheets [34][35][36][37][38][39] . Midgley 34 introduced the "Distillation Gym", an environment in which distillation trains for non-azeotropic mixtures are generated by a soft-actor-critic RL agent and simulated in the opensource process simulator COCO.…”