Hierarchical framework integrating rapidly-exploring random tree with deep reinforcement learning for autonomous vehicle

This paper proposes a systematic driving framework where the decision making module of reinforcement learning (RL) is integrated with rapidly-exploring random tree (RRT) as motion planning. RL is used to generate local goals and semantic speed commands to control the longitudinal speed of a vehicle while rewards are designed for the driving safety and the traffic efficiency. Guaranteeing the driving comfort, RRT returns a feasible path to be followed by the vehicle with the speed commands. The scene decomposition approach is implemented to scale the deep neural network (DNN) to environments with multiple traffic participants and double deep Q-networks (DDQN) with prioritized experience replay (PER) is utilized to accelerate the training process. To handle the disturbance of the perception of the agent, we use an ensemble of neural networks to evaluate the uncertainty of decisions. It has shown that the proposed framework can tackle unexpected actions of traffic participants at an intersection yielding safe, comfort and efficient driving behaviors. Also, the ensemble of DDQN with PER is proved to be superior over standard DDQN in terms of learning efficiency and disturbance vulnerability.