Avaliação de Dependabilidade e Segurança em Arquiteturas Serverless-LLM através de Injeção Dinâmica de Falhas
Resumo
A convergência entre arquiteturas serverless e IA Generativa cria novas superfícies de ataque, exigindo validação rigorosa. Este trabalho avalia a dependabilidade e segurança em ambientes Serveless-LLM via CIMut, ferramenta que utiliza LLMs para automatizar a injeção de falhas semânticas em tempo de execução (Monkey Patching) na AWS. Ao avaliar a interação entre vulnerabilidades de código e defesas dos modelos, os resultados indicam 100% de eficácia na injeção, expondo riscos de exaustão financeira e vazamento em logs. Como achado crítico, a ’Defesa Emergente’ do modelo mitigou falhas de código, mas cedeu a Prompt Injections estruturais, evidenciando a fragilidade na fronteira entre a aplicação e a IA.Referências
Amazon Web Services (2021). AWS Fault Injection Service: User Guide. AWS Documentation.
Basiri, A., Behnam, N., de Rooij, R., Hochstein, L., Kosewski, L., Reynolds, J., and Rosenthal, C. (2016). Chaos engineering. IEEE Software, 33(3):35–41.
Casola, V., Benedictis, A. D., Rak, M., and Villano, U. (2020). A security-by-design methodology for serverless computing. IEEE Transactions on Cloud Computing.
Chao, P. et al. (2024). Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419.
Chen, S. et al. (2024). Struq: Defending against prompt injection with structured queries. arXiv preprint arXiv:2402.06363.
Cloud Native Computing Foundation (2024). Cncf annual survey 2023. Technical report, Cloud Native Computing Foundation.
Datadog (2023). State of serverless 2023. Technical report, Datadog Research.
Deng, J. et al. (2024). Large language models for software engineering: A systematic literature review. arXiv preprint arXiv:2308.10620.
Derczynski, L. et al. (2024). garak: A generator of harmful responses for probing llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), pages 1–9.
Greshake, K. et al. (2023). Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec), pages 79–90.
Hines, K. et al. (2024). Defending against indirect prompt injection attacks with spotlighting. arXiv preprint arXiv:2403.14720.
Hovmöller, A. (2023). Mutmut: Mutation testing system for python. GitHub Repository/PyPI.
Jia, Y. and Harman, M. (2011). An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5):649–678.
Khan, T. et al. (2024). The code quality of generative ai: A systematic review. IEEE Access, 12:1234–1256.
Khanfir, A. et al. (2023). Efficient mutation testing via pre-trained language models. arXiv preprint arXiv:2301.03542.
National Institute of Standards and Technology (2019). Security strategies for microservices-based application systems. Technical Report Special Publication 800-204, NIST.
OWASP Foundation (2023). OWASP Top 10 for Large Language Model Applications. OWASP Project, version 1.1 edition.
Pusuluri, V. S. R. (2022). Taxonomy of security and privacy issues in serverless computing. Master’s thesis, St. Cloud State University. Culminating Projects in Information Assurance, Vol. 120.
Webster, I. (2023). Promptfoo: Cli for testing, evaluating, and red-teaming llms. GitHub Repository/Documentation.
Wei, A., Haghtalab, N., and Steinhardt, J. (2023). Jailbroken: How does llm safety training fail? In Advances in Neural Information Processing Systems (NeurIPS).
Basiri, A., Behnam, N., de Rooij, R., Hochstein, L., Kosewski, L., Reynolds, J., and Rosenthal, C. (2016). Chaos engineering. IEEE Software, 33(3):35–41.
Casola, V., Benedictis, A. D., Rak, M., and Villano, U. (2020). A security-by-design methodology for serverless computing. IEEE Transactions on Cloud Computing.
Chao, P. et al. (2024). Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419.
Chen, S. et al. (2024). Struq: Defending against prompt injection with structured queries. arXiv preprint arXiv:2402.06363.
Cloud Native Computing Foundation (2024). Cncf annual survey 2023. Technical report, Cloud Native Computing Foundation.
Datadog (2023). State of serverless 2023. Technical report, Datadog Research.
Deng, J. et al. (2024). Large language models for software engineering: A systematic literature review. arXiv preprint arXiv:2308.10620.
Derczynski, L. et al. (2024). garak: A generator of harmful responses for probing llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), pages 1–9.
Greshake, K. et al. (2023). Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec), pages 79–90.
Hines, K. et al. (2024). Defending against indirect prompt injection attacks with spotlighting. arXiv preprint arXiv:2403.14720.
Hovmöller, A. (2023). Mutmut: Mutation testing system for python. GitHub Repository/PyPI.
Jia, Y. and Harman, M. (2011). An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5):649–678.
Khan, T. et al. (2024). The code quality of generative ai: A systematic review. IEEE Access, 12:1234–1256.
Khanfir, A. et al. (2023). Efficient mutation testing via pre-trained language models. arXiv preprint arXiv:2301.03542.
National Institute of Standards and Technology (2019). Security strategies for microservices-based application systems. Technical Report Special Publication 800-204, NIST.
OWASP Foundation (2023). OWASP Top 10 for Large Language Model Applications. OWASP Project, version 1.1 edition.
Pusuluri, V. S. R. (2022). Taxonomy of security and privacy issues in serverless computing. Master’s thesis, St. Cloud State University. Culminating Projects in Information Assurance, Vol. 120.
Webster, I. (2023). Promptfoo: Cli for testing, evaluating, and red-teaming llms. GitHub Repository/Documentation.
Wei, A., Haghtalab, N., and Steinhardt, J. (2023). Jailbroken: How does llm safety training fail? In Advances in Neural Information Processing Systems (NeurIPS).
Publicado
19/07/2026
Como Citar
DUARTE, Guilherme Silva; SOUSA, Erica Teixeira Gomes de.
Avaliação de Dependabilidade e Segurança em Arquiteturas Serverless-LLM através de Injeção Dinâmica de Falhas. In: SIMPÓSIO DE INFRAESTRUTURA DIGITAL/NUVEM PARA PESQUISA (PESQUISA@NUVEM), 1. , 2026, Gramado/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 20-28.
DOI: https://doi.org/10.5753/pesquisanuvem.2026.21055.
