An LLM-Based Agentic Pipeline for Generator-Level Adversarial Evaluation of Smart Grid IDSs

• Camilla B. Quincozes UNIPAMPA
• Guilherme C. Mundt UNIPAMPA
• Ian Rankin University of Pittsburgh
• Silvio E. Quincozes UNIPAMPA
• Paulo S. Severo UNIPAMPA
• Daniel Mossé University of Pittsburgh

DOI: https://doi.org/10.5753/sbseg.2026.29283

Resumo

Intrusion Detection Systems (IDSs) for Smart Grids are often evaluated using fixed datasets, which may hide weaknesses against variations of known attacks. This paper proposes an agentic pipeline for parametric adversarial evaluation of IDSs in IEC-61850 environments. The pipeline integrates the ERENO synthetic dataset generator, a Random Forest-based IDS, and Large Language Model (LLM)-based agents that iteratively modify parameters of Masquerade attack scenarios. Instead of perturbing samples directly, the approach operates at the dataset-generation level by changing attack configuration files. We analyze both traditional metrics (e.g., F1-score and recall) and execution indicators (e.g., skipped iterations, failed executions, and degenerate-variant flags). Our results indicate that the proposed LLM-guided agentic architecture can generate attack variants that decrease the intrusion detection performance (e.g., by increasing false negatives).

Referências

Acharya, R., Al Sardy, L., Muhammad, M., and German, R. (2026). Advis-g: An adversarially defended intrusion detection system for smart grids using deep learning. KI-Künstliche Intelligenz, pages 1–23.

Aftab, M. A., Hussain, S. S., Ali, I., and Ustun, T. S. (2020). Iec 61850 based substation automation system: A survey. International Journal of Electrical Power & Energy Systems, 120:106008.

Alenezi, M. (2026). From prompt-response to goal-directed systems: The evolution of agentic ai software architecture. arXiv preprint arXiv:2602.10479.

Asimopoulos, D. C., Radoglou-Grammatikis, P., Makris, I., Mladenov, V., Psannis, K. E., Goudos, S., and Sarigiannidis, P. (2023). Breaching the defense: Investigating fgsm and ctgan adversarial attacks on iec 60870-5-104 ai-enabled intrusion detection systems. In Proceedings of the 18th International Conference on Availability, Reliability and Security, ARES ’23, New York, NY, USA. Association for Computing Machinery.

Brunner, C. (2008). Iec 61850 for power system communication. In 2008 IEEE/PES Transmission and Distribution Conference and Exposition, pages 1–6. IEEE.

de Oliveira, J. A., dos Santos, A. F. P., and Salles, R. M. (2025). Rnn for intrusion detection in digital substations based on the iec 61850. Journal of Information Security and Applications, 94:104197.

Hu, X., Chen, H., Bao, H., Wang, W., Liu, F., Zhou, G., and Yin, P. (2024). A llm-based agent for the automatic generation and generalization of ids rules. In 2024 IEEE 23rd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), pages 1875–1880.

Jaffal, N. O., Alkhanafseh, M., and Mohaisen, D. (2025). Large language models in cybersecurity: A survey of applications, vulnerabilities, and defense techniques. AI, 6(9):216.

Liao, H.-J., Richard Lin, C.-H., Lin, Y.-C., and Tung, K.-Y. (2013). Intrusion detection system: A comprehensive review. Journal of Network and Computer Applications, 36(1):16–24.

Mustafa, A., Khan, M. T., Umer, M. A., Masood, Z., and Ahmed, C. M. (2025). Adversarial sample generation for anomaly detection in industrial control systems. In Proceedings of the 1st Workshop on Modeling and Verification for Secure and Performant Cyber-Physical Systems, pages 1–7.

Mutua, N. M., Nadjm-Tehrani, S., and Matoušek, P. (2025). Penetrating the power grid: Realistic adversarial attacks on smart grid intrusion detection systems. In Oliva, G., Panzieri, S., Hämmerli, B., Pascucci, F., and Faramondi, L., editors, Critical Information Infrastructures Security, pages 249–268, Cham. Springer Nature Switzerland.

Piccialli, F., Chiaro, D., Sarwar, S., Cerciello, D., Qi, P., and Mele, V. (2025). Agentai: A comprehensive survey on autonomous agents in distributed ai for industry 4.0. Expert Systems with Applications, 291:128404.

Qiao, Y., Sathyanarayana, N. B., Shi, C., He, Z., Wang, T., and Hou, T. (2026). A survey on adversarial machine learning: Attacks, defenses, real-world applications, and future research directions. Neurocomputing, page 132670.

Quincozes, S. E., Albuquerque, C., Passos, D., and Mossé, D. (2023). Ereno: A framework for generating realistic iec–61850 intrusion detection datasets for smart grids. IEEE Transactions on Dependable and Secure Computing, 21(4):3851–3865.

Resende, P. A. A. and Drummond, A. C. (2018). A survey of random forest based methods for intrusion detection systems. ACM Computing Surveys (CSUR), 51(3):1–36.

Sahani, N., Zhu, R., Cho, J.-H., and Liu, C.-C. (2023). Machine learning-based intrusion detection for smart grid computing: A survey. ACM Trans. Cyber-Phys. Syst., 7(2).

Sidhu, T. S. and Yin, Y. (2007). Modelling and simulation for performance evaluation of iec61850-based substation communication systems. IEEE Transactions on Power Delivery, 22(3):1482–1489.

Teryak, H., Albaseer, A., Abdallah, M., Al-Kuwari, S., and Qaraqe, M. (2023). Double-edged defense: Thwarting cyber attacks and adversarial machine learning in iec 60870-5-104 smart grids. IEEE Open Journal of the Industrial Electronics Society, 4:629–642.

Ustun, T. S., Hussain, S. M. S., Ulutas, A., Onen, A., Roomi, M. M., and Mashima, D. (2021). Machine learning-based intrusion detection for achieving cybersecurity in smart grids using iec 61850 goose messages. Symmetry, 13(5).

Zhai, W., Liao, J., Chen, Z., Su, B., and Zhao, X. (2025). A survey of task planning with large language models. Intelligent Computing, 4:0124.