Reinforcement learning vs. rule-based adaptive traffic signal control: A Fourier basis linear function approximation for traffic signal control

Abstract

Reinforcement learning is an efficient, widely used machine learning technique that performs well when the state and action spaces have a reasonable size. This is rarely the case regarding control-related problems, as for instance controlling traffic signals. Here, the state space can be very large. In order to deal with the curse of dimensionality, a rough discretization of such space can be employed. However, this is effective just up to a certain point. A way to mitigate this is to use techniques that generalize the state space such as function approximation. In this paper, a linear function approximation is used. Specifically, SARSA ( λ ) with Fourier basis features is implemented to control traffic signals in the agent-based transport simulation MATSim. The results are compared not only to trivial controllers such as fixed-time, but also to state-of-the-art rule-based adaptive methods. It is concluded that SARSA ( λ ) with Fourier basis features is able to outperform such methods, especially in scenarios with varying traffic demands or unexpected events.

Financial Accounting Information Data Analysis System Based on Internet of Things

“…With the increasing competition, many enterprises begin to pay more and more attention to the ne management reform based on "accurate decision-making, accurate planning, accurate control, and accurate assessment." How to transform massive data into valuable information in a timely and e ective manner, how to give full play to the maximum value of enterprise information, so that managers can have an insight into the huge economic bene ts brought by data management solutions and e ectively help enterprises realize information strategies have become the most concerned issues of many enterprise managers [1,2]. Business intelligence is another important application in the eld of enterprise management software after ERP (enterprise resource planning).…”