Using Large Language Models to Classify Test Case Complexity with Explainability - Custódio, Tiago; Carvalho, André; Santos, Maikon; Soares, Yan; Melo, Hallyson; Ferreira, Nikson; Marques, Rodrigo (Anais do Simpósio Brasileiro de Engenharia de Software (SBES); 2025: Anais do XXXIX Simpósio Brasileiro de Engenharia de Software, 2025)
The classification of black-box test case complexity is a key task in software testing, enabling resource prioritization and the identification of edge scenarios. In this work, we propose a three-stage LLM-based pipeline that integrates explanation generation into the classification process, treating justifications as central to model decision-making. Our approach formulates the task as a conditional generation problem, where LLMs are guided to first produce a rationale and then a complexity label. Experimental results demonstrate that this strategy improves both predictive accuracy and explainability compared to using LLMs directly for classification. We show that LLM-generated justifications not only enhance user trust but also contribute to more consistent and explainable decisions.