RRT-QX Real-Time Kinodynamic Motion Planning in Dynamic Environments with Continuous-Time Reinforcement Learning

This chapter presents a real-time kinodynamic motion planning technique for linear systems with completely unknown dynamics in environments with unpredictable obstacles. The methodology incorporates: i) a sampling-based algorithm for path planning and fast replanning; and ii) continuous-time Q-learning for the solution of finite-horizon optimal control problems in real-time. The path planner produces a set of waypoints that dynamically change in time according to the unpredictably appearing obstacles, while the Q-learning controller is responsible for optimal waypoint navigation. The efficacy of the methodology has been validated with simulations. This chapter presents a real-time kinodynamic motion planning technique for linear systems with completely unknown dynamics in environments with unpredictable obstacles. Substantial improvements in artificial intelligence, computing resources, and software tools have enabled tremendous capabilities to mobile robots and autonomous systems. The problem of navigation is a core topic in robotics and autonomous vehicles, as the majority of robotic applications require safe path planning and obstacle avoidance. The problem of kinodynamic motion planning is introduced. Kinodynamic RRT employs the dynamical model of the system, but the proposed control strategy is selected either randomly or by testing multiple controls and selecting the best. The environment is completely unknown and consists of obstacles that appear throughout the navigation. Since the environment is unknown, we suppose there are no obstacles other than the obstacles detected in the perception range. The simulations reveal that the autonomous rover can efficiently perform safe navigation with no collisions in an unknown dynamic domain.