On-Premise vs. Cloud: Local LLMs for Vulnerability Extraction from Security Scanner Reports

• Beatriz Machado UNIPAMPA https://orcid.org/0009-0002-2750-0323
• Cristhian Kapelinski UNIPAMPA https://orcid.org/0009-0005-5750-022X
• Diego Kreutz UNIPAMPA https://orcid.org/0000-0003-0830-0238

DOI: https://doi.org/10.5753/sbseg_estendido.2026.29879

Resumo

Cloud LLMs extract structured data from vulnerability-scanner reports accurately, but each report maps an organization’s attack surface, so routing it to a third-party API trades confidentiality for that accuracy. We evaluate nine local models (4B to 21B) against a DeepSeek cloud reference on three ground-truth baselines. The best local model, on a consumer GPU, trails the cloud by only 1.2 points in free-text fidelity and matches it on structured fields; the spread across local models far exceeds this gap, making model choice the decisive factor for private, on-premise extraction.

Referências

Aghaei, E., Niu, X., Shadid, W., and Al-Shaer, E. (2022). SecureBERT: A domain-specific language model for cybersecurity. arXiv preprint arXiv:2204.02685.

Chen, L. and Varoquaux, G. (2024). What is the role of small models in the LLM era: A survey. arXiv preprint arXiv:2409.06857.

Ferrag, M. A., Alwahedi, F., Battah, A., Cherif, B., Mechri, A., Tihanyi, N., Bisztray, T., and Debbah, M. (2025). Generative AI in cybersecurity: A comprehensive review of LLM applications and vulnerabilities. Internet of Things and Cyber-Physical Systems.

Keltek, M., Hu, R., Fani Sani, M., and Li, Z. (2024). Boosting cybersecurity vulnerability scanning based on LLM-supported static application security testing. arXiv preprint arXiv:2409.15735.

Li, G., Zhang, Y., Wang, Y., Yan, S., Wang, L., and Wei, T. (2025). PRIV-QA: Privacy-preserving question answering for cloud large language models. arXiv preprint arXiv:2502.13564.

Lu, Z., Li, X., Cai, D., Yi, R., Liu, F., Zhang, X., Lane, N. D., and Xu, M. (2024). Small language models: Survey, measurements, and insights. arXiv preprint arXiv:2409.15790.

Machado, B., Lautert, D., Kapelinski, C., and Kreutz, D. (2025). Structured extraction of vulnerabilities in openvas and tenable was reports using llms. In Anais da XXII Escola Regional de Redes de Computadores, pages 144–150, Porto Alegre, RS, Brasil. SBC.

Machado, B., Lautert, D., Kapelinski, C., Kreutz, D., Ferrão, I. G., and Bof, A. (2026). MulitaMiner: A multi-version evaluation of LLM-based vulnerability report extraction. In Anais do XXVI Simpósio Brasileiro de Cibersegurança (SBSeg), Porto Alegre, RS, Brasil. SBC.

Rahman, F. I., Halim, S. M., Singhal, A., and Khan, L. (2024). ALERT: A framework for efficient extraction of attack techniques from cyber threat intelligence reports using active learning. In Data and Applications Security and Privacy XXXVIII (DBSec). Springer.

Wiest, I. C., Ferber, D., Zhu, J., van Treeck, M., Meyer, S. K., Juglan, R., Carrero, Z. I., Paech, D., Kleesiek, J., Ebert, M. P., Truhn, D., and Kather, J. N. (2023). From text to tables: A local privacy preserving large language model for structured information retrieval from medical documents. medRxiv.

Yan, B., Li, K., Xu, M., Dong, Y., Zhang, Y., Ren, Z., and Cheng, X. (2024). On protecting the data privacy of large language models (LLMs): A survey. arXiv preprint arXiv:2403.05156.

Yang, L., Zhu, H., Mu, J., and Li, Q. (2026). KGAgent4CTI: Unlocking the power of LLM in threat intelligence. Cybersecurity, 9(76).

Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. (2020). BERTScore: Evaluating text generation with BERT. In International Conference on Learning Representations (ICLR).