DAST-AI: Reducing False Positives in API Security Testing via LLM-based Semantic Analysis

• Gabriel F. R. de Melo UFPE
• Divanilson R. Campelo UFPE

DOI: https://doi.org/10.5753/sbseg.2026.27119

Resumo

Dynamic Application Security Testing (DAST) is a critical methodology for identifying vulnerabilities in web applications. However, traditional DAST tools suffer from high false positive rates, causing alert fatigue among security teams. This issue is particularly acute in RESTful APIs regarding authorization flaws—such as Broken Object Level Authorization (BOLA) and missing authentication—where distinguishing between a legitimately public endpoint and a vulnerable one requires semantic context. This paper introduces DAST-AI, a hybrid security testing framework that integrates Large Language Models (LLMs) as a post-execution semantic oracle to analyze HTTP response payloads and determine whether sensitive data is improperly exposed. The evaluation is scoped to the zero-credential black-box scenario—testing APIs without any prior authentication—applicable to external security assessments and unauthenticated CI/CD pipelines. A controlled ablation study against two deliberately vulnerable benchmarks—OWASP Juice Shop (Node.js, 68 endpoints) and VAmPI (Python/Flask, 12 endpoints)—isolates the contribution of the LLM semantic oracle: removing it yields a syntax-based baseline (F1=0.49 on Juice Shop; F1=0.857 on VAmPI), while adding it produces DAST-AI (F1=0.80; F1=0.923), reducing false positives by 86.4% and 50.0% respectively, at a negligible cost of $0.000037 per endpoint. An authenticated reference comparison—simulating the ZAP Access Control Scanner methodology with valid credentials—scores F1=0.28 on Juice Shop and F1=0.857 on VAmPI (identical to the unauthenticated baseline), demonstrating that credentials alone do not resolve the problem: semantic reasoning about response content is the key differentiating factor.

Referências

0x4m4 (Muhammad Osama) (2025). HexStrike AI: MCP agents for autonomous AI-driven penetration testing. GitHub. [link]. Accessed: July 2026.

Arcuri, A., Sahin, O., and Zhang, M. (2025). Fuzzing for detecting access policy violations in REST APIs. In Proceedings of the 36th IEEE International Symposium on Software Reliability Engineering (ISSRE). IEEE.

Atlidakis, V., Godefroid, P., and Polishchuk, M. (2019). RESTler: Stateful REST API fuzzing. In Proceedings of the 41st International Conference on Software Engineering (ICSE), pages 748–758. IEEE/ACM.

Barreto, C. B. L. (2025). SonarAutoFix: Automatização da correção de código com base em análise estática de código. B.s. thesis, Centro de Informática, Universidade Federal de Pernambuco, Recife, Brazil.

erev0s (2021). VAmPI: The vulnerable API (based on OWASP top 10). GitHub. [link].

Felderer, M., Büchler, M., Johns, M., Brucker, A. D., Breu, R., and Pretschner, A. (2016). Security testing: A survey. In Advances in Computers, volume 101, pages 1–51. Elsevier.

Ferrag, M. A. et al. (2025). SecureFalcon: Are we there yet in automated software vulnerability detection with LLMs? arXiv preprint arXiv:2307.06616.

HackerAI (2025). HackerAI: Autonomous AI penetration testing platform. [link]. Accessed: July 2026.

Jain, N. et al. (2023). StaticFixer: From static analysis to static repair. arXiv preprint arXiv:2307.12465.

Kimminich, B. (2025). Pwning OWASP juice shop. Companion guide. [link]. Accessed: July 2026.

OWASP Foundation (2010). OWASP ZAP: Zed attack proxy — the world’s most widely used web application scanner. [link].

OWASP Foundation (2023). OWASP API Security Top 10 – 2023. Open Web Application Security Project. [link]. Accessed: July 2026.

OWASP Foundation (2024). OFFAT: OFFensive API Tester — automated API security testing tool for detecting BOLA, authorization bypass, and injection vulnerabilities. GitHub. [link].

OWASP Foundation (2025). OWASP Top 10:2025 – the ten most critical web application security risks. Open Web Application Security Project. [link]. Accessed: July 2026.

OWASP Foundation and Kimminich, B. (2025). OWASP Juice Shop: Probably the most modern and sophisticated insecure web application. GitHub. [link]. Challenge catalog: data/static/challenges.yml. Accessed: July 2026.

Pasca, E. M., Erdei, R., Delinschi, D., and Matei, O. (2025). Enhancing API security testing against BOLA and authentication vulnerabilities through an LLM-enhanced framework. In Soft Computing Models in Industrial and Environmental Applications (SOCO 2024), pages 231–240. Springer.

ProjectDiscovery (2020). Nuclei: Fast and customizable vulnerability scanner based on simple YAML based DSL. GitHub. [link].

Thool, A. U. (2026). Bridging Security and Agility: A Comprehensive Approach to Integrating Security Practices in Agile Development through DAST, LLMs, & Automation. PhD thesis, Virginia Tech, Blacksburg, VA, USA.

Zhang, X. et al. (2025). Unlocking the next generation of REST API security: A critical analysis of the path from fuzzing to LLMs. In 2025 IEEE 10th International Conference on Data Science in Cyberspace (DSC). IEEE.

Zhang, Y. et al. (2023). How well does LLM generate security tests? arXiv preprint arXiv:2310.00710.

Zheng, T. et al. (2024). RESTLess: Enhancing state-of-the-art REST API fuzzing with LLMs in cloud service computing. IEEE Transactions on Services Computing, 17(6):4225–4239.