Kill Chain Catalyst for Autonomous Red Team Operations in Dynamic Attack Scenarios
Resumo
From the perspective of real-world cyber attacks, executing actions with minimal failures and steps is crucial to reducing the likelihood of exposure. Although research on autonomous cyber attacks predominantly employs Reinforcement Learning (RL), this approach has gaps in scenarios such as limited training data and low resilience in dynamic environments. Therefore, the Kill Chain Catalyst (KCC) has been introduced: an RL algorithm that employs decision tree logic, inspired by genetic alignment, prioritizing resilience in dynamic scenarios and limited experiences. Experiments reveal significant improvements in reducing steps and failures, as well as increased rewards when using KCC compared to other RL algorithms.Referências
Al-Azzawi, M., Doan, D., Sipola, T., Hautamäki, J., and Kokkonen, T. (2024). Artificial intelligence cyberattacks in red teaming: A scoping review. In World Conference on Information Systems and Technologies, pages 129–138. Springer.
Breiman, L. (2001). Random forests. Machine learning, 45:5–32.
Che Mat, N. I., Jamil, N., Yusoff, Y., and Mat Kiah, M. L. (2024). A systematic literature review on advanced persistent threat behaviors and its detection strategy. Journal of Cybersecurity, 10(1):tyad023.
Chen, J., Hu, S., Zheng, H., Xing, C., and Zhang, G. (2023). Gail-pt: An intelligent penetration testing framework with generative adversarial imitation learning. Computers Security, 126:103055.
Disha, R. A. and Waheed, S. (2022). Performance analysis of machine learning models for intrusion detection system using gini impurity-based weighted random forest (giwrf) feature selection technique. Cybersecurity, 5(1):1.
Farouk, M., Sakr, R. H., and Hikal, N. (2024). Identifying the most accurate machine learning classification technique to detect network threats. Neural Computing and Applications, 36(16):8977–8994.
Gancheva, V. and Stoev, H. (2023). An algorithm for pairwise dna sequences alignment. In International Work-Conference on Bioinformatics and Biomedical Engineering, pages 48–61. Springer.
Gangupantulu, R., Cody, T., Rahma, A., Redino, C., Clark, R., and Park, P. (2021). Crown jewels analysis using reinforcement learning with attack graphs. In 2021 IEEE Symposium Series on Computational Intelligence (SSCI), pages 1–6.
Holm, H. (2022). Lore a red team emulation tool. IEEE Transactions on Dependable and Secure Computing, 1:1–1.
Horta Neto, A. J., dos Santos, A. F. P., and Goldschmidt, R. R. (2024). Evaluating the stealth of reinforcement learning-based cyber attacks against unknown scenarios using knowledge transfer techniques. Journal of Computer Security, (Preprint):1–19.
Ibrahim, M. K., Yusof, U. K., Eisa, T. A. E., and Nasser, M. (2024). Bioinspired algorithms for multiple sequence alignment: A systematic review and roadmap. Applied Sciences, 14(6):2433.
Janisch, J., Pevnỳ, T., and Lisỳ, V. (2023). Nasimemu: Network attack simulator & emulator for training agents generalizing to novel scenarios. In European Symposium on Research in Computer Security, pages 589–608. Springer.
Li, L., El Rami, J.-P. S., Taylor, A., Rao, J. H., and Kunz, T. (2022). Enabling a network ai gym for autonomous cyber agents. In 2022 International Conference on Computational Science and Computational Intelligence (CSCI), pages 172–177. IEEE.
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016). Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pages 1928–1937. PMLR.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015). Human-level control through deep reinforcement learning. nature, 518(7540):529–533.
Ortiz-Garces, I., Gutierrez, R., Guerra, D., Sanchez-Viteri, S., and Villegas-Ch., W. (2023). Development of a platform for learning cybersecurity using capturing the flag competitions. Electronics, 12(7).
Paudel, B. and Amariucai, G. (2023). Reinforcement learning approach to generate zero-dynamics attacks on control systems without state space models. In European Symposium on Research in Computer Security, pages 3–22. Springer.
Poinsignon, T., Poulain, P., Gallopin, M., and Lelandais, G. (2023). Working with omics data: An interdisciplinary challenge at the crossroads of biology and computer science. In Machine Learning for Brain Disorders, pages 313–330. Springer.
Pozdniakov, K., Alonso, E., Stankovic, V., Tam, K., and Jones, K. (2020). Smart security audit: Reinforcement learning with a deep neural network approximator. In 2020 International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA), pages 1–8.
Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015). Trust region policy optimization. In International conference on machine learning, pages 1889–1897. PMLR.
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms.
Standen, M., Lucas, M., Bowman, D., Richer, T. J., Kim, J., and Marriott, D. (2021). Cyborg: A gym for the development of autonomous cyber agents. In IJCAI-21 1st International Workshop on Adaptive Cyber Defense. arXiv.
Sutton, R. S. and Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press, second edition.
Tran, K., Akella, A., Standen, M., Kim, J., Bowman, D., Richer, T., and Lin, C.-T. (2021). Deep hierarchical reinforcement agents for automated penetration testing. In IJCAI-21 1st International Workshop on Adaptive Cyber Defense. arXiv.
Yang, Y. and Liu, X. (2022). Behaviour-diverse automatic penetration testing: A curiosity-driven multi-objective deep reinforcement learning approach.
Zhou, S., Liu, J., Hou, D., Zhong, X., and Zhang, Y. (2021). Autonomous penetration testing based on improved deep q-network. Applied Sciences, 11(19).
Breiman, L. (2001). Random forests. Machine learning, 45:5–32.
Che Mat, N. I., Jamil, N., Yusoff, Y., and Mat Kiah, M. L. (2024). A systematic literature review on advanced persistent threat behaviors and its detection strategy. Journal of Cybersecurity, 10(1):tyad023.
Chen, J., Hu, S., Zheng, H., Xing, C., and Zhang, G. (2023). Gail-pt: An intelligent penetration testing framework with generative adversarial imitation learning. Computers Security, 126:103055.
Disha, R. A. and Waheed, S. (2022). Performance analysis of machine learning models for intrusion detection system using gini impurity-based weighted random forest (giwrf) feature selection technique. Cybersecurity, 5(1):1.
Farouk, M., Sakr, R. H., and Hikal, N. (2024). Identifying the most accurate machine learning classification technique to detect network threats. Neural Computing and Applications, 36(16):8977–8994.
Gancheva, V. and Stoev, H. (2023). An algorithm for pairwise dna sequences alignment. In International Work-Conference on Bioinformatics and Biomedical Engineering, pages 48–61. Springer.
Gangupantulu, R., Cody, T., Rahma, A., Redino, C., Clark, R., and Park, P. (2021). Crown jewels analysis using reinforcement learning with attack graphs. In 2021 IEEE Symposium Series on Computational Intelligence (SSCI), pages 1–6.
Holm, H. (2022). Lore a red team emulation tool. IEEE Transactions on Dependable and Secure Computing, 1:1–1.
Horta Neto, A. J., dos Santos, A. F. P., and Goldschmidt, R. R. (2024). Evaluating the stealth of reinforcement learning-based cyber attacks against unknown scenarios using knowledge transfer techniques. Journal of Computer Security, (Preprint):1–19.
Ibrahim, M. K., Yusof, U. K., Eisa, T. A. E., and Nasser, M. (2024). Bioinspired algorithms for multiple sequence alignment: A systematic review and roadmap. Applied Sciences, 14(6):2433.
Janisch, J., Pevnỳ, T., and Lisỳ, V. (2023). Nasimemu: Network attack simulator & emulator for training agents generalizing to novel scenarios. In European Symposium on Research in Computer Security, pages 589–608. Springer.
Li, L., El Rami, J.-P. S., Taylor, A., Rao, J. H., and Kunz, T. (2022). Enabling a network ai gym for autonomous cyber agents. In 2022 International Conference on Computational Science and Computational Intelligence (CSCI), pages 172–177. IEEE.
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016). Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pages 1928–1937. PMLR.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015). Human-level control through deep reinforcement learning. nature, 518(7540):529–533.
Ortiz-Garces, I., Gutierrez, R., Guerra, D., Sanchez-Viteri, S., and Villegas-Ch., W. (2023). Development of a platform for learning cybersecurity using capturing the flag competitions. Electronics, 12(7).
Paudel, B. and Amariucai, G. (2023). Reinforcement learning approach to generate zero-dynamics attacks on control systems without state space models. In European Symposium on Research in Computer Security, pages 3–22. Springer.
Poinsignon, T., Poulain, P., Gallopin, M., and Lelandais, G. (2023). Working with omics data: An interdisciplinary challenge at the crossroads of biology and computer science. In Machine Learning for Brain Disorders, pages 313–330. Springer.
Pozdniakov, K., Alonso, E., Stankovic, V., Tam, K., and Jones, K. (2020). Smart security audit: Reinforcement learning with a deep neural network approximator. In 2020 International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA), pages 1–8.
Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015). Trust region policy optimization. In International conference on machine learning, pages 1889–1897. PMLR.
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms.
Standen, M., Lucas, M., Bowman, D., Richer, T. J., Kim, J., and Marriott, D. (2021). Cyborg: A gym for the development of autonomous cyber agents. In IJCAI-21 1st International Workshop on Adaptive Cyber Defense. arXiv.
Sutton, R. S. and Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press, second edition.
Tran, K., Akella, A., Standen, M., Kim, J., Bowman, D., Richer, T., and Lin, C.-T. (2021). Deep hierarchical reinforcement agents for automated penetration testing. In IJCAI-21 1st International Workshop on Adaptive Cyber Defense. arXiv.
Yang, Y. and Liu, X. (2022). Behaviour-diverse automatic penetration testing: A curiosity-driven multi-objective deep reinforcement learning approach.
Zhou, S., Liu, J., Hou, D., Zhong, X., and Zhang, Y. (2021). Autonomous penetration testing based on improved deep q-network. Applied Sciences, 11(19).
Publicado
16/09/2024
Como Citar
HORTA, Antonio; SANTOS, Anderson dos; GOLDSHMIDT, Ronaldo.
Kill Chain Catalyst for Autonomous Red Team Operations in Dynamic Attack Scenarios. In: SIMPÓSIO BRASILEIRO DE SEGURANÇA DA INFORMAÇÃO E DE SISTEMAS COMPUTACIONAIS (SBSEG), 24. , 2024, São José dos Campos/SP.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 415-430.
DOI: https://doi.org/10.5753/sbseg.2024.241371.
