Approximate Optimal Trajectory Tracking With Sparse Bellman Error Extrapolation

This article provides an approximate online adaptive solution to the infinite-horizon optimal tracking problem for control-affine continuous-time nonlinear systems with uncertain drift dynamics. A model-based approximate dynamic programming (ADP) approach, which is facilitated using a concurrent learning-based system identifier, approximates the optimal value function. To reduce the computational complexity of model-based ADP, the state space is segmented into user-defined segments (i.e., regions). Off-policy trajectories are selected within each segment to facilitate learning of the value function weight estimates; this process is called Bellman error (BE) extrapolation. Within certain segments of the state space, sparse neural networks are used to reduce the computational expense of BE extrapolation. Discontinuities occur in the weight update laws since different groupings of extrapolated BE trajectories are active in certain regions of the state space. A Lyapunov-like stability analysis is presented to prove boundedness of the overall system in the presence of discontinuities. Simulation results are included to demonstrate the performance and validity of the developed method. The simulation results demonstrate that using the sparse, switched BE extrapolation method developed in this article reduces the computation time by 85.6% when compared to the traditional BE extrapolation method.