Reinforcement Learning (RL) has been recognized as one of the most effective methods to optimize traffic signal control. However, due to the inappropriate design of RL elements (i.e., reward and state) for complex traffic dynamics, existing RL-based approaches suffer from slow convergence to optimal traffic signal plans. Meanwhile, to simplify the traffic modeling, most optimization methods assume that the phase duration of traffic signals is constant, which strongly limits the RL capability to search for traffic signal control policies with shorter average vehicle travel time and better GreenWave control. To address these issues, this project proposes a novel intensity- and phase duration-aware RL method named IPDALight for the optimization of traffic signal control. Inspired by the Max Pressure (MP)-based traffic control strategy used in the transportation field, we introduce a new concept named intensity, which ensures that our reward design and state representation can accurately reflect the status of vehicles. By taking the coordination of neighboring intersections into account, our approach enables the fine-tuning of phase duration of traffic signals to adapt to dynamic traffic situations. Comprehensive experimental results on both synthetic and real-world traffic scenarios show that, compared with the state-of-the-art RL methods, IPDALight can not only achieve better average vehicle travel time and greenwave control for various multi-intersection scenarios, but also converge to optimal solutions much faster.

• Python 3.5+
• cityflow
• tqdm
• tensorflow 1.11.0
• keras
• pandas
• numpy

Our experiments are implemented on top of the traffic simulator Cityflow. Detailed installation guide files can be found in https://cityflow-project.github.io/

sudo apt update && sudo apt install -y build-essential cmake

git clone https://github.com/cityflow-project/CityFlow.git

pip install .

import cityflow

eng = cityflow.Engine

• train.py

  The main file of experiments. Choosing the dataset, setting simulation parameters and starting the train.

• dqn_agent.py

  Implement RL agent for proposed IPDALight.

• cityflow_env.py

  Define a simulator environment to interact with the simulator and obtain needed data (e.g., state, action and reward).

• utility.py

  Some functions for experiments (e.g., reading information from roadnet file and plot the travel time convergence).

For the experiments, we used both synthetic and realworld traffic datasets provided by https://traffic-signal-control.github.io/dataset.html.

• synthetic data

  We considered four synthetic traffic datasets with different scales (i.e., 1x3, 2x2, 3x3, 4x4). Traffic files can be found in data/template_lsr/new/1_3 and data/template_lsr/new/2_2 and data/template_lsr/new/3_3 and data/template_lsr/new/4_4.

• real-world data

We used two datasets collected from the real-world traffic of two cities (i.e., Hangzhou and Jinan) in China via roadside surveillance cameras. Traffic files of Hangzhou can be found in data/hangzhou, which contains 16 intersections in the form of a 4x4 grid. Traffic files of Jinan can be found in data/jinan, which contains 12 intersections in the form of a 3x4 grid.

Detailed README for expriment of comparison of average travel time and convergence rate TravelTimeComparison

Detailed README for expriment of comparison of fairness FairnessComparison

Detailed README for expriment of comparison of intersection pressure PressureComparison

Detailed README for expriment of comparison of greenwave control effects GreenwaveComparison