We consider the problem of fully decentralized multi-agent reinforcement learning (MARL), where the agents are located at the nodes of a time-varying communication network. Specifically, we assume that the reward functions of the agents might correspond to different tasks, and are only known to the corresponding agent. Moreover, each agent makes individual decisions based on both the information observed locally and the messages received from its neighbors over the network.Within this setting, the collective goal of the agents is to maximize the globally averaged return over the network through exchanging information with their neighbors. To this end, we propose two decentralized actor-critic algorithms with function approximation, which are applicable to large-scale MARL problems where both the number of states and the number of agents are massively large. Under the decentralized structure, the actor step is performed individually by each agent with no need to infer the policies of others. For the critic step, we propose a consensus update via communication over the network. Our algorithms are fully incremental and can be implemented in an online fashion. Convergence analyses of the algorithms are provided when the value functions are approximated within the class of linear functions. Extensive simulation results with both linear and nonlinear function approximations are presented to validate the proposed algorithms. Our work appears to be the first study of fully decentralized MARL algorithms for networked agents with function approximation, with provable convergence guarantees.

No abstract

No abstract

Background/Aims: The therapeutic efficacy of paclitaxel is hampered by chemotherapeutic resistance in non-small cell lung cancer (NSCLC). Rsf-1 enhanced paclitaxel resistance via nuclear factor-κB (NF-κB) in ovarian cancer cells and nasopharyngeal carcinoma. This study assessed the function of Rsf-1 in the modulation of the sensitivity of NSCLC to paclitaxel via the NF-κB pathway. Methods: The mRNA and protein levels of the related genes were quantified by RT-PCR and Western blotting. Rsf-1 silencing was achieved with CRISPR/Cas9 gene editing. Cell cycle, migration and proliferation were tested with flow cytometry, transwell test and CCK8 test. Cell apoptosis was analyzed with flow cytometry and quantification of C-capase3. The parameters of the tumors were measured in H460 cell xenograft mice. Results: Rsf-1 was highly expressed in H460 and H1299 cells. Rsf-1 knockout caused cell arrest at the G1 phase, increased cell apoptosis, and decreased migration and cell proliferation. Rsf-1 knockout increased the inhibition of cell proliferation, the reduction in cell migration and the augment in cell apoptosis in paclitaxel treated H460 and H1299 cells. Rsf-1 knockout further enhanced the paclitaxel-mediated decrease in the volume and weight of the tumors in H460 cell xenograft mice. Helenalin and Rsf-1 knockout decreased the protein levels of p-P65, BcL2, CFLAR, and XIAP; hSNF2H knockout decreased the protein level of NF-κB p-P65 without altering Rsf-1 and p65 protein levels, while Rsf-1 and hSNF2H double knockout decreased the level of NF-κB p-P65, in H1299 and H460 cells. Conclusion: These results demonstrate that Rsf-1 influences the sensitivity of NSCLC to paclitaxel via regulation of the NF-κB pathway and its downstream genes.