Stabilization of Phasor Measurement Sensor-Based Markovian Jump CPSs Through Soft Actor-Critic

The advent of phasor measurement sensors (PMSs) made a revolution to transition from conventional power systems to cyber-physical systems (CPSs), where measurement signals are transmitted through communication channels. While these smart sensors provide real-time monitoring, control, and protection of smart grid systems, their vulnerability to the time-delay switch (TDS) attack in the transmission channels imposes significant challenges on the stability of networked systems. The Markovian jump theory is a particular type of stochastic process that can adopted to describe time-delay attacks in many practical systems. In this article, the random TDS attack is modeled by Markov chains in a specific CPS system with PMS. In order to make the Markovian jump more realistic, this work discusses the stabilization issues for a particular class of continuous-time interconnected power plants, in which the transition rate (probability) is typically constrained. PMS sensor devices are responsible for sending measurement signals of system outputs to the communication network. To ensure the quality of system output, the soft actor-critic (SAC) based on maximum entropy learning is developed to optimize the controller coefficients for stabilizing the CPS problem. By training the actor and critic neural networks, the SAC-agent tries to adaptively adjust the control parameters. Compared with the prevalent state-of-the-art schemes, a higher level of stability can be obtained with the application of the proposed schemes (realized by entropy learning), which confirms its feasibility and usefulness in dealing with the Markovian jump-based systems.