Robust H∞ tracking of linear discrete‐time systems using Q‐learning

This paper deals with a robust H∞$$ {H}_{\infty } $$ tracking problem with a discounted factor. A new auxiliary system is established in terms of norm‐bounded time‐varying uncertainties. It is shown that the robust discounted H∞$$ {H}_{\infty } $$ tracking problem for the auxiliary system solves the original problem. Then, the new robust discounted H∞$$ {H}_{\infty } $$ tracking problem is represented as a well‐known zero‐sum game problem. Moreover, the robust tracking Bellman equation and the robust tracking Algebraic Riccati equation (RTARE) are inferred. A lower bound of a discounted factor for stability is obtained to assure the stability of the closed‐loop system. Based on the auxiliary system, the system is reshaped in a new structure that is applicable to Reinforcement Learning methods. Finally, an online Q‐learning algorithm without the knowledge of system matrices is proposed to solve the algebraic Riccati equation associated with the robust discounted H∞$$ {H}_{\infty } $$ tracking problem for the auxiliary system. Simulation results are given to verify the effectiveness and merits of the proposed method.