Evaluating the Potential of Large Language Models in Security-Related Software Requirements Classification - Martin, Murilo; Coutinho, Daniel; Uchôa, Anderson; Pereira, Juliana Alves (Anais do Simpósio Brasileiro de Engenharia de Software (SBES); 2025: Anais do XXXIX Simpósio Brasileiro de Engenharia de SoftwareDO - 10.5753/sbes.2025.9935, 2025)
An effective classification of security-related software requirements is crucial to mitigate potential threats and ensure robust system design. This study investigates the performance of Large Language Models (LLMs) in classifying security-related requirements compared to traditional Machine Learning (ML) methods. Using the SecReq, DOSSPRE and PROMISE+ datasets, we evaluated ten LLMs across various prompt engineering strategies. The results demonstrate that LLMs achieve high accuracy and outperform traditional ML-based models in several evaluation scenarios and that prompt engineering can significantly enhance the model’s ability to identify security-related requirements. This work underscores the domaingeneralization capabilities of LLMs and their potential to streamline requirements classification without the complexity of feature engineering or dataset-specific fine-tuning often required by ML-based approaches. Researchers, practitioners, and tool developers can leverage these findings to advance automated approaches in security requirements engineering.