Deep Reinforcement Learning for Voltage Control in Power Systems

• Mauricio W. Barg Radix Engineering and Software Development / PUC-RJ https://orcid.org/0000-0003-2172-1627
• Barbara S. Rodrigues Radix Engineering and Software Development https://orcid.org/0009-0003-9540-666X
• Gabriela T. Justino Radix Engineering and Software Development https://orcid.org/0000-0001-6154-3525
• Kleyton Pontes Cotta Radix Engineering and Software Development https://orcid.org/0000-0003-4332-5641
• Hugo R. V. Portuita Radix Engineering and Software Development https://orcid.org/0009-0004-2205-1287
• Flávio L. Loução Jr. Radix Engineering and Software Development https://orcid.org/0009-0009-1615-3283
• Iran Pereira Abreu Real Time Operation / ISA-CTEEP https://orcid.org/0009-0007-5909-9550
• Antônio Carlos Pigossi Jr. Real Time Operation / ISA-CTEEP https://orcid.org/0009-0005-7690-0122

Resumo

The great complexity and size of electrical power systems makes their operation and control a challenging task. Maintaining a stable voltage profile to assure the security and stability of the system is one of many tasks that must be conducted daily by power system operators and its automatic control equipment. This work proposes a deep reinforcement learning framework for controlling the equipment responsible for keeping the voltages across the system buses within their limits. More specifically, a smart agent that is capable of deciding the best course of action in order to keep the system’s voltages within a specified range while taking into account system’s conditions is proposed. Besides the traditional deep reinforcement learning approach, three novel reinforcement learning variations named windowed, ensemble and windowed ensemble Q-Learning, which alter the agent’s learning process for voltage control, are presented and tested on IEEE 13, 37 and 123 bus systems, simulated on OpenDSS.