Online and Robust Intermittent Motion Planning in Dynamic and Changing Environments

In this article, we propose RRT-Q ^{\textrm{\,X}}_{\infty} , an online and intermittent kinodynamic motion planning framework for dynamic environments with unknown robot dynamics and unknown disturbances. We leverage RRT ^{\textrm{\,X}} for global path planning and rapid replanning to produce waypoints as a sequence of boundary-value problems (BVPs). For each BVP, we formulate a finite-horizon, continuous-time zero-sum game, where the control input is the minimizer, and the worst case disturbance is the maximizer. We propose a robust intermittent Q-learning controller for waypoint navigation with completely unknown system dynamics, external disturbances, and intermittent control updates. We execute a relaxed persistence of excitation technique to guarantee that the Q-learning controller converges to the optimal controller. We provide rigorous Lyapunov-based proofs to guarantee the closed-loop stability of the equilibrium point. The effectiveness of the proposed RRT-Q ^{\textrm{\,X}}_{\infty} is illustrated with Monte Carlo numerical experiments in numerous dynamic and changing environments.