LLMs e Engenharia de Prompt para Classificação Automatizada de Incidentes em SOCs
Resumo
Neste trabalho, avaliamos o uso de LLMs e engenharias de prompt para automatizar a classificação de incidentes de segurança em SOCs, buscando acelerar a resposta e aprimorar a qualidade das ações (e.g., seleção de playbooks), o que pode reduzir o tempo de resposta das empresas. Testamos três estratégias de prompting (PHP, SHP, HTP) em quatro LLMs (Gemini 2, GPT-4, LLaMA 4 e Grok 3) usando dados reais de CSIRTs e SOCs de empresas brasileiras. Nossos resultados indicam que o GEMINI alcançou 92,27% de acurácia em relação às classificações humanas ao usar PHP, enquanto as outras combinações de técnicas e LLMs demonstraram alguma variabilidade, o que pode afetar sua confiabilidade em cenários sensíveis.Referências
CERT.br (2025). Incidentes notificados ao cert.br. [link].
Chen, J., Tian, J., and Jin, Y. (2024). Self-hint prompting improves zero-shot reasoning in large language models via reflective cycle. In Proceedings of the 46th Annual CCSS.
Google (2024). Modelos gemini na vertex ai. [link].
Ibrishimova, M. D. (2019). Cyber incident classification: Issues and challenges. In Xhafa, F., Leu, F.-Y., Ficco, M., and Yang, C.-T., editors, Advances on P2P, Parallel, Grid, Cloud and Internet Computing, pages 469–477. Springer International Publishing.
Li, Y., Tian, J., He, H., and Jin, Y. (2024). Hypothesis testing prompting improves deductive reasoning in large language models. arXiv preprint arXiv:2405.06707.
Meta (2024). Llama 4: Advancing multimodal intelligence. [link].
Nasution, A. H., Monika, W., Onan, A., and Murakami, Y. (2025). Benchmarking 21 open-source large language models for phishing link detection with prompt engineering. Information, 16(5):366.
Nelson, A., Rekhi, S., Souppaya, M., and Scarfone, K. (2025). Incident response recommendations and considerations for cybersecurity risk management: A csf 2.0 community profile. Technical Report NIST SP 800-61r3, NIST.
Ogundairo, O. and Broklyn, P. (2024). Natural language processing for cybersecurity incident analysis. Journal of Cyber Security.
OpenAI (2024). Gpt-4o mini. [link].
xAI (2024). Grok-3: Next-generation model. [link].
Zheng, Liu, X. et al. (2023). Progressive-hint prompting improves reasoning in large language models.
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. (2022). Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations.
Chen, J., Tian, J., and Jin, Y. (2024). Self-hint prompting improves zero-shot reasoning in large language models via reflective cycle. In Proceedings of the 46th Annual CCSS.
Google (2024). Modelos gemini na vertex ai. [link].
Ibrishimova, M. D. (2019). Cyber incident classification: Issues and challenges. In Xhafa, F., Leu, F.-Y., Ficco, M., and Yang, C.-T., editors, Advances on P2P, Parallel, Grid, Cloud and Internet Computing, pages 469–477. Springer International Publishing.
Li, Y., Tian, J., He, H., and Jin, Y. (2024). Hypothesis testing prompting improves deductive reasoning in large language models. arXiv preprint arXiv:2405.06707.
Meta (2024). Llama 4: Advancing multimodal intelligence. [link].
Nasution, A. H., Monika, W., Onan, A., and Murakami, Y. (2025). Benchmarking 21 open-source large language models for phishing link detection with prompt engineering. Information, 16(5):366.
Nelson, A., Rekhi, S., Souppaya, M., and Scarfone, K. (2025). Incident response recommendations and considerations for cybersecurity risk management: A csf 2.0 community profile. Technical Report NIST SP 800-61r3, NIST.
Ogundairo, O. and Broklyn, P. (2024). Natural language processing for cybersecurity incident analysis. Journal of Cyber Security.
OpenAI (2024). Gpt-4o mini. [link].
xAI (2024). Grok-3: Next-generation model. [link].
Zheng, Liu, X. et al. (2023). Progressive-hint prompting improves reasoning in large language models.
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. (2022). Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations.
Publicado
01/09/2025
Como Citar
SEVERO, Alex Sandre Pinheiro; LAUTERT, Douglas Paim; ALMEIDA, Gefté Alcantara de; KREUTZ, Diego; RODRIGO, Godinho; PEREIRA JR, Lourenco A.; BERTHOLDO, Leandro M..
LLMs e Engenharia de Prompt para Classificação Automatizada de Incidentes em SOCs. In: TRILHA DE INTERAÇÃO COM A INDÚSTRIA E DE INOVAÇÃO - SIMPÓSIO BRASILEIRO DE CIBERSEGURANÇA (SBSEG), 25. , 2025, Foz do Iguaçu/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 447-455.
DOI: https://doi.org/10.5753/sbseg_estendido.2025.12510.
