Three-Stage Inverter-Based Peak Shaving and Volt-VAR Control in Active Distribution Networks Using Online Safe Deep Reinforcement Learning

This paper presents a three-stage inverter-based peak shaving and Volt-VAR control (VVC) framework in active distribution systems using the online safe deep reinforcement learning (DRL) method. The proposed framework aims to reduce the peak load, voltage violations, and real power loss by coordinating three stages with different control timescales. In the first stage, a day-ahead charging/discharging scheduling of energy storage systems (ESSs) with a 30 min resolution is performed via their inverters for peak shaving. In the second stage, the discharging power of ESSs is adjusted through measurements with a 1 min resolution to completely shave peak loads. A model-free DRL algorithm integrated with a safety module is also implemented in the second stage. Using this algorithm, the reactive powers of photovoltaic (PV) systems and ESSs are controlled by the DRL agent to reduce the voltage violation and real power loss, whereas no voltage violation occurs during the online training process. In the third stage, a proportional-integral controller with real-power compensation is integrated into inverters of PV systems and ESSs to rapidly mitigate local voltage violations with a 0.1 s resolution. The high efficiency and safety of the proposed method were validated on the IEEE 33-bus and IEEE 123-bus systems.