Deep Reinforcement Learning for Battery Arbitrage with Price and Renewable Data

• Gabrielly de Q. Pereira UEPG
• Marcella Scoczynski UTFPR
• Adriana A. Guimarães UTFPR
• Isabelly dos S. Pacheco UEPG
• James W. Freitas UdeG
• Diego Oliva UEPG
• Felipe S. Silva UEPG

DOI: https://doi.org/10.5753/eniac.2026.29408

Resumo

Este artigo propõe uma abordagem baseada em Proximal Policy Optimization (PPO) para arbitragem horária de energia com baterias utilizando dados abertos da plataforma Open Power System Data. O modelo utiliza preços do mercado do dia seguinte, carga, geração renovável e dados meteorológicos da Alemanha e da zona Alemanha–Luxemburgo para aprender decisões contínuas de carga e descarga sob restrições operacionais da bateria. Utilizando conjuntos cronológicos de treinamento, validação e teste, o PPO foi comparado com uma heurística baseada em quantis de preço. A política proposta alcançou lucros de 730,17 EUR na validação e 1.019,15 EUR no teste, frente a 510,95 EUR e 491,78 EUR da referência. Os resultados demonstram o potencial do PPO para o despacho de baterias orientado a dados em mercados de eletricidade do dia seguinte.

Referências

Cao, J., Harrold, D., Fan, Z., Morstyn, T., Healey, D., and Li, K. (2020). Deep reinforcement learning-based energy storage arbitrage with accurate lithium-ion battery degradation model. IEEE Transactions on Smart Grid, 11(5):4513–4521.

Chen, Q., Kuang, Z., Liu, X., and Zhang, T. (2024). Application-oriented assessment of grid-connected pv-battery system with deep reinforcement learning in buildings considering electricity price dynamics. Applied Energy, 364:123163.

Harrold, D. J. B., Cao, J., and Fan, Z. (2022a). Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning. Energy, 238:121958.

Harrold, D. J. B., Cao, J., and Fan, Z. (2022b). Renewable energy integration and micro-grid energy trading using multi-agent deep reinforcement learning. Applied Energy, 318:119151.

Jeong, J., Kim, S. W., and Kim, H. (2023). Deep reinforcement learning based real-time renewable energy bidding with battery control. IEEE Transactions on Energy Markets, Policy and Regulation, 1(2):85–96.

Jiang, X., Tang, F., Chen, M., Li, B., Yu, Y., Ding, J., and Li, X. (2025). A reinforcement learning-based approach with downside-risk protection for battery dispatch in day-ahead markets. International Transactions on Electrical Energy Systems.

Miao, Y., Chen, T., Bu, S., Liang, H., and Han, Z. (2021). Co-optimizing battery storage for energy arbitrage and frequency regulation in real-time markets using deep reinforcement learning. Energies, 14(24):8365.

Open Power System Data (2020a). Data package time series. DOI: 10.25832/time_series/2020-10-06. Version 2020-10-06.

Open Power System Data (2020b). Data package weather data. DOI: 10.25832/weather_data/2020-09-16. Version 2020-09-16.

Open Power System Data (2020c). Data platform. Accessed: 2026-05-05.

Sage, M., Campbell, J., and Zhao, Y. F. (2024). Enhancing battery storage energy arbitrage with deep reinforcement learning and time-series forecasting. In Proceedings of the ASME 2024 Energy Sustainability Conference, page V001T01A007.

Samende, C., Fan, Z., and Cao, J. (2022). Battery and hydrogen energy storage control in a smart energy network with flexible energy demand using deep reinforcement learning. arXiv preprint arXiv:2208.12779.

Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.

Xu, G., Shi, J., Wu, J., Lu, C., Wu, C., Wang, D., and Han, Z. (2024). An optimal solutions-guided deep reinforcement learning approach for online energy storage control. Applied Energy, 361:122915.

Zhang, H., Zhang, G., Zhao, M., and Liu, Y. (2024). Load forecasting-based learning system for energy management with battery degradation estimation: A deep reinforcement learning approach. IEEE Transactions on Consumer Electronics, 70(1):2342–2352.