No abstract

We have developed a graph-based Variational Autoencoder with Gaussian Mixture hidden space (GraphGMVAE), a deep learning approach for controllable magnitude of scaffold hopping in generative chemistry. It can effectively and accurately generate molecules from a given reference compound, with excellent scaffold novelty against known molecules in the literature or patents (97.9% are novel scaffolds). Moreover, a pipeline for prioritizing the generated compounds was also proposed to narrow down our validation focus. In this work, GraphGMVAE was validated by rapidly hopping the scaffold from FDA-approved upadacitinib, which is an inhibitor of human Janus kinase 1 (JAK1), to generate more potent molecules with novel chemical scaffolds. Seven compounds were synthesized and tested to be active in biochemical assays. The most potent molecule has 5.0 nM activity against JAK1 kinase, which shows that the GraphGMVAE model can design molecules like how a human expert does but with high efficiency and accuracy.

In the proton exchange membrane fuel cell (PEMFC) system, the flow of air and hydrogen is the main factor influencing the output characteristics of PEMFC, and there is a coordination problem between their flow controls. Thus, the integrated controller of the PEMFC gas supply system based on distributed deep reinforcement learning (DDRL) is proposed to solve this problem, it combines the original airflow controller and hydrogen flow controller into one. Besides, edge-cloud collaborative multiple tricks distributed deep deterministic policy gradient (ECMTD-DDPG) algorithm is presented. In this algorithm, an edge exploration policy is adopted, suggesting that the edge explores including DDPG, soft actor-critic (SAC), and conventional control algorithm are employed to realize distributed exploration in the environment, and a classified experience replay mechanism is introduced to improve exploration efficiency. Moreover, various tricks are combined with the cloud centralized training policy to address the overestimation of Q-value in DDPG. Ultimately, a model-free integrated controller of the PEMFC gas supply system with better global searching ability and training efficiency is obtained. The simulation verifies that the controller enables the flows of air and hydrogen to respond more rapidly to the changing load.

This paper proposes a multi-objective integrated automatic generation control (MOI-AGC) that combines a controller with a dispatch together. This can contribute to improving both control performance and economy in a power grid with multiple continuous power disturbances. Subsequently, a distributed classification replay twin delayed deep deterministic policy gradient (DCR-TD3) is designed for MOI-AGC. On the one hand, DCR-TD3 introduces the classification replay method based on multiple explorers with different parameter actor networks for distributed optimization. On the other hand, the optimal control strategy is obtained through DCR-TD3 in an extremely random environment based on frequency deviation, regional control error together with frequency mileage payment as the reward function. This helps address the problem of frequency instability caused by multiple stochastic disturbance in a grid with a large number of distributed energies. Simulation verification is performed for the two-area load frequency control (LFC) model, with the result showing that the proposed algorithm has better control performance and economic benefits. Besides, compared with the existing algorithms, it can achieve a regional optimum control, reducing frequency mileage payment. INDEX TERMS performance-based frequency regulation market; multi-objective integrated automatic generation control; distributed classification replay twin delayed deep deterministic policy gradient; regulation mileage payment; multiple continuous power disturbances

No abstract

In this paper, an adaptive Proportion integration (PI) controller for varying the output voltage of a proton exchange membrane fuel cell (PEMFC) is proposed. The PI controller operates on the basis of ambient intelligence large-scale deep reinforcement learning. It functions as a coefficient tuner based on an ambient intelligence exploration multi-delay deep deterministic policy gradient (AIEM-DDPG) algorithm. This algorithm is an improvement on the original deep deterministic police gradient (DDPG) algorithm, which incorporates ambient intelligence exploration. The DDPG algorithm serves as the core, and the AIEM-DDPG algorithm runs on a variety of deep reinforcement learning algorithms, including soft actorcritic (SAC), deep deterministic policy gradient (DDPG), proximal policy optimization (PPO) and double deep Q-network (DDQN) algorithms, to attain distributed exploration in the environment. In addition, a classified priority experience replay mechanism is introduced to improve the exploration efficiency. Clipping multi-Q learning, policy delayed updating, target policy smooth regularization and other methods are utilized to solve the problem of Q-value overestimation. A model-free algorithm with good global searching ability and optimization speed is demonstrated. Simulation results show that the AIEM-DDPG adaptive PI controller attains better robustness and adaptability, as well as a good control effect.INDEX TERMS distributed deep reinforcement learning; ambient intelligence exploration multi-delay deep deterministic policy gradient; proton exchange membrane fuel cell; air mass flow control; intelligent controller

This paper proposes a distributed hierarchical automatic generation control (AGC) framework with multiple regulation units in the performance-based frequency regulation market, named virtual generation alliance automatic generation control (VGA-AGC), aiming to achieve the coordination of control algorithm and AGC dispatch algorithm and adapt to the development trend of AGC from centralized framework to centralized-decentralized framework. The framework also involves a multi agent distributed multiple improved deep deterministic policy gradient (MADMI-TD3) algorithm that is characterized by excellent global search capability and optimizing speed. The algorithm can help create an optimal AGC strategy in a randomization environment so as to obtain an optimal cooperative control of AGC. According to a simulation verification on the LFC model for an interconnected power grid of a province, the algorithm is superior to the current algorithms and conventional engineering methods in terms of control performance and economic benefits. In other words, the algorithm can improve control performance and reduce the regulation mileage payment. INDEX TERMS performance-based frequency regulation market; virtual generation alliance automatic generation control (VGA-AGC); multi agent distributed multiple improved deep deterministic policy gradient; regulation mileage payment; centralized-decentralized autonomy

No abstract