Increasing Test Coverage by Automating BDD Tests in Proofs of Concepts (POCs) using LLM

• Shexmo Richarlison Ribeiro dos Santos UFS
• Raiane Eunice S. Fernandes UFS
• Marcos Cesar Barbosa dos Santos UFS
• Michel S. Soares UFS
• Fabio Gomes Rocha SafeLabs / UFS
• Sabrina Marczak PUCRS

Resumo

In today’s landscape, software manufacturers must deliver quickly and with high quality to remain competitive, especially in the cybersecurity sector. Recognizing this need, we have recently implemented several strategies to accelerate our time to market without compromising quality. We introduced the Proof of Concept (POC) and Proof of Value (POV) stages in the Enterprise Architecture team before initiating inception processes for Minimum Viable Products (MVP) and new features. Initially, these proofs focused only on concept validation. However, since 2023, due to the growing need for rapid and high-quality innovation, POCs/POVs have begun to include robust implementations. We adopted the Behaviour-Driven Development (BDD) approach to define user stories and acceptance criteria, which provided a solid evaluation of POC/POV quality and involved the implementation team from the outset. To prevent products from incurring technical debt, we implemented AutoDevSuite, which uses LLM to generate tests based on user stories and acceptance criteria automatically. We used AutoDevSuite in a POC/POV of a cybersecurity product, and the results showed a significant expansion in test coverage, aligned with the acceptance criteria, demonstrating the tool’s effectiveness in automating and improving test quality.

Palavras-chave: Artificial Intelligence (AI), Automatic test code generator, Behavior-Driven Development (BDD), Large Language Model (LLM), Software quality