Towards SLMs Usage in Contextualized Question Generation for Security Information Audits: A Study Using the G-Eval Framework - Souza, Vinícius F.; Gomes, Diego R.; Mota, Rafael P. B.; Aires, Fernando (Anais Estendidos do Simpósio Brasileiro de Cibersegurança (SBSeg); 2026: Anais Estendidos do XXVI Simpósio Brasileiro de Cibersegurança, 2026)
Conducting information security audits today is no trivial task. One key aspect is the quantity and quality of controls to be selected for the assessment, especially considering different types of organizations. This paper evaluates the ability of Small Language Models (SLMs) with Chain-of-Thought to adapt generic questions from the ISO/IEC 27001:2022 and 27002:2022 control lists to specific organizational contexts (hospital, bank, retail, development, and government). Five SLMs were tested using an automatic evaluator based on the G-Eval framework (1–5 scale) and human validation (n=25), revealing that intermediate models (3B–8B) achieve acceptable quality (mean ≥ 4.0). Agreement between the automatic evaluator and human evaluators varied by up to 1.2 points in some dimensions, and limitations were also observed in Portuguese. The results suggest feasibility for supporting security audits with human supervision.