Electrochemically mediated redox-active processes are gaining momentum as a promising liquid-phase separation technology. Compared to conventional systems, they offer potential benefits, such as smaller energy footprints, nondestructive operation, reversibility, and tunability for specific analyte removal, with clear applications to societal and industrial challenges like water treatment and chemical synthesis. An asymmetric Faradaic cell heterogeneously functionalized with a metallopolymer at the anode and a hexacyanoferrate material at the cathode is presented for the first time. The redox-active species' iron centers enhance the electrosorption of heavy metal oxyanions with up to 98% removal in the ppb range, and offer tunable operating windows as low as ≈0.1 V at ≈1 A m −2 . By avoiding water splitting, the hexacyanoferrate cathode imparts additional advantages, namely a fourfold reduction in adsorption energy requirements, full suppression of solution pH increase, and the ability to capture redox-active catalytic anions such as polyoxometalates without altering their bulk oxidation state. This hybrid framework of a polymeric ferrocene anode and crystalline hexacyanoferrate cathode allows for simultaneous and synergistic uptake of anions and cations, respectively, creating a new asymmetric scheme for water-based separations, with foreseeable future extension to fields such as ion-sensing, energy storage, and electrocatalysis.

Many existing traffic signal controllers are either simple adaptive controllers based on sensors placed around traffic intersections, or optimized by traffic engineers on a fixed schedule. Optimizing traffic controllers is time consuming and usually require experienced traffic engineers. Recent research has demonstrated the potential of using deep reinforcement learning (DRL) in this context. However, most of the studies do not consider realistic settings that could seamlessly transition into deployment. In this paper, we propose a DRL-based adaptive traffic signal control framework that explicitly considers realistic traffic scenarios, sensors, and physical constraints. In this framework, we also propose a novel reward function that shows significantly improved traffic performance compared to the typical baseline pre-timed and fully-actuated traffic signals controllers. The framework is implemented and validated on a simulation platform emulating real-life traffic scenarios and sensor data streams.

No abstract

No abstract

Robustness of Deep Reinforcement Learning (DRL) algorithms towards adversarial attacks in real world applications such as those deployed in cyber-physical systems (CPS) are of increasing concern. Numerous studies have investigated the mechanisms of attacks on the RL agent's state space. Nonetheless, attacks on the RL agent's action space (corresponding to actuators in engineering systems) are equally perverse, but such attacks are relatively less studied in the ML literature. In this work, we first frame the problem as an optimization problem of minimizing the cumulative reward of an RL agent with decoupled constraints as the budget of attack. We propose the white-box Myopic Action Space (MAS) attack algorithm that distributes the attacks across the action space dimensions. Next, we reformulate the optimization problem above with the same objective function, but with a temporally coupled constraint on the attack budget to take into account the approximated dynamics of the agent. This leads to the white-box Look-ahead Action Space (LAS) attack algorithm that distributes the attacks across the action and temporal dimensions. Our results showed that using the same amount of resources, the LAS attack deteriorates the agent's performance significantly more than the MAS attack. This reveals the possibility that with limited resource, an adversary can utilize the agent's dynamics to malevolently craft attacks that causes the agent to fail. Additionally, we leverage these attack strategies as a possible tool to gain insights on the potential vulnerabilities of DRL agents.

No abstract

No abstract

A traffic signal is a fundamental part of the traffic control system to reduce congestion and enhance safety. Since the inception of motorized vehicles, traffic signal controllers are put in place to coordinate and maintain traffic flow. With the number of vehicles on the road increasing exponentially, it is imperative to innovate new traffic control frameworks to cope with the high-density traffic demand. In this regard, recent advances in machine/deep learning have enabled significant progress towards reducing congestion using reinforcement learning for traffic signal control. However, most of these works are still not ready for deployment due to assumptions of perfect knowledge of the traffic environment. In reality, congestion detection or prediction systems are at best able to approximate the traffic state with significant noise. In this work, we propose a robust training framework for reinforcement learning agents that can handle such noisy approximation of the traffic states. Specifically, we show that by carefully adding synthetic perturbations to the state space, such as the queue length during training, the reinforcement learning agents can be robustified. Conceptually, our approach is similar to adversarial training schemes and can lead to successful deployment of reinforcement learning agent-based traffic signal controllers.