Convergence analysis and application of fuzzy-HDP for nonlinear discrete-time HJB systems

In this paper, a type of fuzzy system structure is applied to heuristic dynamic programming (HDP) algorithm to solve nonlinear discrete-time Hamilton–Jacobi–Bellman (DT-HJB) problems. The fuzzy system here is adopted as a 0-order T–S fuzzy system using triangle membership functions (MFs). The convergence of HDP and approximability of the multivariate 0-order T–S fuzzy system is analyzed in this paper. It is derived that the cost function and control policy of HDP can be iterated to the DT-HJB solution and optimal policy. The multivariate 0-order T–S (Tanaka–Sugeno) fuzzy system using triangle MFs is proven as a universal approximator, to guarantee the convergence of the Fuzzy-HDP mechanism. Some simulations are implemented to observe the performance of the proposed method both in mathematical solution and practical issue. It is concluded that Fuzzy-HDP outperforms traditional optimal control in more complex systems.