Imitative Reinforcement Learning Fusing Vision and Pure Pursuit for Self-driving

Abstract

Autonomous urban driving navigation is still an open problem and has ample room for improvement in unknown complex environments and terrible weather conditions. In this paper, we propose a two-stage framework, called IPP-RL, to handle these problems. IPP means an Imitation learning method fusing visual information with the additional steering angle calculated by Pure-Pursuit (PP) method, and RL means using Reinforcement Learning for further training. In our IPP model, the visual information captured by camera can be compensated by the calculated steering angle, thus it could perform well under bad weather conditions. However, imitation learning performance is limited by the driving data severely. Thus we use a reinforcement learning method-Deep Deterministic Policy Gradient (DDPG)-in the second stage training, which shares the learned weights from pretrained IPP model. In this way, our IPP-RL can lower the dependency of imitation learning on demonstration data and solve the problem of low exploration efficiency caused by randomly initialized weights in reinforcement learning. Moreover, we design a more reasonable reward function and use the n-step return to update the critic-network in DDPG. Our experiments on CARLA driving benchmark demonstrate that our IPP-RL is robust to lousy weather conditions and shows remarkable generalization capability in unknown environments on navigation task.