Graph Reinforcement Learning for Carbon-Aware Electric Vehicles in Power-Transport Networks

Abstract

Transitioning towards a low-carbon future necessitates massive efforts from both the transport and power sectors. Electric vehicles (EVs) have emerged as a promising approach to realize this objective, leveraging their smart routing strategies and vehicle-to-grid (V2G) techniques. Previous studies have addressed various challenges in EV routing and scheduling through model-based optimization methods while ignoring the system uncertainties and dynamics. This paper focuses on studying the carbon-aware EV joint routing and scheduling problem within a coupled power-transport network that can enable EV recharging behaviors within the transport network while concurrently delivering carbon-intensity services within the power network. Specifically, a carbon emission flow model is introduced as a mechanism for tracing and calculating the nodal carbon intensity signals tailored for EVs to provide their carbon services. To solve this problem, we propose a model-free multi-agent reinforcement learning method that harnesses graph convolutional networks to capture essential network features and employs a parameter-sharing framework to learn large-scale control policies. The efficacy and scalability of the proposed method in achieving cost-effective and low-carbon transitions are verified through case studies involving two power-transport networks with 100 and 1,000 EVs, respectively.