A Note on State Parameterizations in Output Feedback Reinforcement Learning Control of Linear Systems

This note presents an analysis of the state parameterizations used in output feedback reinforcement learning (RL) control. Output feedback algorithms based on state parameterization involve additional conditions on the state parameterization beyond the standard conditions on the system matrices for their convergence to the optimal solution. It is shown that the state parameterization matrix needs to be of full row rank to guarantee the convergence of the output feedback RL algorithms. We present conditions in terms of the system matrices and the user-defined observer dynamics that ensure full row rank of the state parameterization matrix.