Assessing the Security of “Vibe Coding”: An Analysis of Vulnerabilities in Web Applications Generated by LLMs

• Leandro L. Retzlaff URI
• Alan Motta URI

DOI: https://doi.org/10.5753/errc.2026.36080

Resumo

Vibe coding shifts validation from source code review to observable behaviour, which may allow vulnerabilities that do not affect perceived functionality to go unnoticed. This paper compares the vulnerability surface of four e-commerce web applications generated by four distinct LLMs from a single specification without additional security requirements. Each application was scanned twice with OWASP ZAP 2.17.0, once per access profile, under identical configuration, and complemented by a manual audit of access control and session management. The scans reported between 32 and 58 unique findings per model and up to nine high-severity findings. Manual reproduction refuted all 18 high-severity findings, the most pronounced difference between models. The defects common to all four applications and not reported by the scanner concern the session lifecycle: tokens remain valid after logout, and signing secrets are embedded in the source code as default values.

Referências

Daniotti, S., Wachs, J., Feng, X., and Neffke, F. (2026). Who is using AI to code? Global diffusion and impact of generative AI. Science, 391(6787):831–835.

Elder, S., Zahan, N., Shu, R., Metro, M., Kozarev, V., Menzies, T., and Williams, L. (2022). Do I really need all this work to find vulnerabilities? Empirical Software Engineering, 27(6):154.

Hadavi, M. A., Bagherdaei, A., and Ghasemi, S. (2021). IDOT: Black-box detection of access control violations in web applications. The ISC International Journal of Information Security, 13(2):117–129.

Hassan, M. A., Shukur, Z., Hasan, M. K., and Al-Khaleefa, A. S. (2020). A review on electronic payments security. Symmetry, 12(8):1344.

Meske, C., Hermanns, T., Von der Weiden, E., Loser, K.-U., and Berger, T. (2025). Vibe coding as a reconfiguration of intent mediation in software development: Definition, implications, and research agenda. IEEE Access, 13:213242–213259.

MITRE Corporation (2023). 2023 CWE Top 25 Most Dangerous Software Weaknesses. [link]. Accessed 13 October 2026.

OWASP Foundation (2021). OWASP Top 10:2021 – The Ten Most Critical Web Application Security Risks. [link]. Accessed 13 October 2026.

Qadir, S., Waheed, E., Khanum, A., and Jehan, S. (2025). Comparative evaluation of approaches & tools for effective security testing of Web applications. PeerJ Computer Science, 11:e2821.

Ramírez, L. C., Limón, X., Sánchez-García, Á. J., and Pérez-Arriaga, J. C. (2024). State of the art of the security of code generated by LLMs: A systematic literature review. In 2024 12th International Conference in Software Engineering Research and Innovation (CONISOFT), pages 331–339. IEEE.

Vargas, F. L. S., Mansilha, R. B., and Kreutz, D. (2026). Security-first evaluation of text-to-terraform: Benchmarking LLMs and SLMs for secure IaC generation. arXiv:2608.02672. Accepted for publication at SBSeg 2026.

ZAP Development Team (2025). ZAP version 2.17.0. [link]. Accessed 13 October 2026.