Intelligent traffic congestion alleviation at intersections based on deep reinforcement learning

Abstract

Traffic congestion is a significant issue in large and growing metropolitan cities. As the bottlenecks of traffic networks, intersections are most prone to be congested and a competent traffic light control scheme plays a crucial role in intelligent transportation systems. Recent studies employing deep reinforcement learning techniques have shown promising results, but they only focus on extracting features from traffic conditions of isolated or neighbouring intersections. In this research, we are the first to propose the utilization of navigation information for traffic signal control, greatly enriching the features for traffic signal control using deep reinforcement learning. In addition, we presented a novel scheme DeepNavi to exploit the temporal-spatial relations from numerous navigation routes and extract dynamic real-time and future traffic features based on deep reinforcement learning, achieving remarkable performance. We tested DeepNavi on a challenging traffic network with 16 intersections, and extensive experiments demonstrated that DeepNavi notably outperforms state-of-the-art baseline methods on different metrics. Simulations showed that even with only part of the navigation routes available in the traffic network, DeepNavi can obtain superior performance, further demonstrating its effectiveness and feasibility.

Then we studied how to control traffic signals for vehicles with special demands, i.e., emergency vehicles. Although emergency vehicles possess the privilege to run a red light, it can be unsafe, and a congested intersection will prevent the exercise of this privilege. When an emergency vehicle arrives, the popular greedy preemption scheme offers a green signal promptly until it leaves the intersection. This guarantees a fast emergency response in most cases. However, this scheme will lead to an adverse impact on vehicles of conflicting directions and may not work when there are other emergency vehicles traveling from conflicting directions simultaneously. Based on deep reinforcement learning, we proposed an intelligent traffic signal control scheme for emergency vehicles, ensuring an expeditious emergency response at congested intersections and alleviating the negative influence on the traffic congestion in conflicting directions. Simulations verified the feasibility and notable effectiveness of our model compared with state-of-the-art baselines over different performance metrics.

Another significant direction for traffic congestion alleviation is making speed plans for vehicles around intersections to efficiently utilize green time resources. The key is to increase the chance of going through green lights and avoid idling at red lights. Existing works fail to fully consider various realistic traffic conditions, such as interfering traffic, free lane switch, and group efficiency, thus compromising effectiveness and limiting feasibility. To address the issues above, we proposed DeepGAL for vehicle control by dividing vehicles into groups, assigning a leader in each group and delivering intelligent control on leaders with deep reinforcement learning. Extensive experiments showed that DeepGAL outperforms state-of-the-art baselines under both light and heavy traffic flows. In addition, simulations on different penetration levels demonstrated that even with only 10% penetration of leader candidates, DeepGAL can notably alleviate traffic congestion at intersections, which validates its feasibility.