Systematic Mapping of Comparative Studies of Static Code Analysis Tools with a Focus on Security

• Francisco Jean S. Sousa UFC
• João Gustavo I. Braga UFC
• Anna Beatriz V. Sousa UFC
• Valéria Lelli L. Dantas UFC
• Rossana M. C. Andrade UFC
• Ismayle S. Santos UECE

DOI: https://doi.org/10.5753/sbseg_estendido.2025.11875

Abstract

Security assurance is a fundamental element in the development process of software solutions, and it is crucial to supervise this aspect throughout the application’s life cycle, starting from the initial idea. Static analysis, applied during the implementation phase, uses automated tools to prevent flaws in the source code. Thus, the choice of static analysis tools that are more effective and comprehensive becomes a critical factor in identifying and correcting issues, such as those related to security. This study conducts a systematic mapping to identify which static application security testing (SAST) tools are best evaluated in comparative studies. Initially, 258 studies were collected, from which 15 were selected, using three different databases as sources, resulting in 64 tools analyzed, covering six programming languages evaluated based on 18 distinct metrics.

References

AlBreiki, H. H. and Mahmoud, Q. H. (2014). Evaluation of static analysis tools for software security. pages 93–98.

Alqaradaghi, M. and Kozsik, T. (2024). Comprehensive evaluation of static analysis tools for their performance in finding vulnerabilities in java code. IEEE Access, 12:55824–55842.

Bennett, G., Hall, T., Winter, E., and Counsell, S. (2024). Semgrep*: Improving the limited performance of static application security testing (sast) tools. page 614–623.

Bhutani, V., Toosi, F. G., and Buckley, J. (2024). Analysing the analysers: An investigation of source code analysis tools. Applied Computer Systems, 29(1):98–111.

Brito, T., Ferreira, M., Monteiro, M., Lopes, P., Barros, M., Santos, J. F., and Santos, N. (2023). Study of javascript static analysis tools for vulnerability detection in node.js packages. IEEE Transactions on Reliability, 72(4):1324–1339.

Charoenwet, W., Thongtanunam, P., Pham, V.-T., and Treude, C. (2024a). An empirical study of static analysis tools for secure code review. page 691–703.

Charoenwet, W., Thongtanunam, P., Pham, V.-T., and Treude, C. (2024b). An empirical study of static analysis tools for secure code review.

Chatzieleftheriou, G. and Katsaros, P. (2011). Test-driving static analysis tools in search of c code vulnerabilities. pages 96–103.

Chess, B. and McGraw, G. (2004). Static analysis for security. IEEE Security Privacy, 2(6):76–79.

Croft, R., Newlands, D., Chen, Z., and Babar, M. A. (2021). An empirical study of rule-based and learning-based approaches for static application security testing. In Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), ESEM ’21, New York, NY, USA. Association for Computing Machinery.

D, V. L. L. (2024). Análise estática. Disciplina Verificação, Validação e Teste de Software, Universidade Federal do Ceará.

de Mendonça, V. R. L., Rodrigues, C. L., de MN Soares, F. A. A., and Vincenzi, A. M. R. (2013). Static analysis techniques and tools: A systematic mapping study. ICSEA.

Díaz, G. and Bermejo, J. R. (2013). Static analysis of source code security: Assessment of tools against samate tests. Information and Software Technology, 55(8):1462–1476.

Esposito, M., Falaschi, V., and Falessi, D. (2024). An extensive comparison of static application security testing tools. page 69–78.

et. al., K. (2004). Vulnerability detection in software applications using static code analysis. Journal of Theoretical and Applied Information TechnologyVolume 102, Issue 4, Pages 1307, 102(4):1307.

ISO/IEC/IEEE 29119-1 (2022). ISO/IEC/IEEE 29119-1:2022 - Software and Systems Engineering — Software Testing — Part 1: Concepts and Definitions. Standard published by ISO/IEC/IEEE. Available at: DOI: 10.1109/ IEEESTD.2022.9698145.

Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., and Linkman, S. (2009). Systematic literature reviews in software engineering – a systematic literature review. Information and Software Technology, 51(1):7–15. Special Section - Most Cited Articles in 2002 and Regular Research Papers.

Kronjee, J., Hommersom, A., and Vranken, H. (2018). Discovering software vulnerabilities using data-flow analysis and machine learning.

Li, K., Chen, S., Fan, L., Feng, R., Liu, H., Liu, C., Liu, Y., and Chen, Y. (2023). Comparison and evaluation on static application security testing (sast) tools for java. page 921–933.

Li, Z., Liu, Z., Wong, W. K., Ma, P., and Wang, S. (2024). Evaluating c/c++ vulnerability detectability of query-based static application security testing tools. IEEE Trans. Dependable Secur. Comput., 21(5):4600–4618.

Moher, D., Liberati, A., Tetzlaff, J., and Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: the prisma statement. Bmj, 339.

Ozturk, O. S., Ekmekcioglu, E., Cetin, O., Arief, B., and Hernandez-Castro, J. (2023). New tricks to old codes: Can ai chatbots replace static code analysis tools? page 13–18.

Pashchenko, I., Dashevskyi, S., and Massacci, F. (2017). Delta-bench: Differential benchmark for static analysis security testing tools. In 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pages 163–168.

SOMMERVILLE, I. (2011). Engenharia de software. tradução ivan bosnic e kalinka g. de o.

Stefanović, D., Nikolić, D., Dakić, D., Spasojević, I., and Ristić, S. (2020). Static code analysis tools: A systematic literature review. In Ann. DAAAM Proc. Int. DAAAM Symp, volume 31, pages 565–573.

Tyagi, S. and Kumar, K. (2018). Evaluation of static web vulnerability analysis tools. pages 1–6.

Zhang, H., Luo, J., Hu, M., Yan, J., Zhang, J., and Qiu, Z. (2023). Detecting exception handling bugs in c++ programs. page 1084–1096.

Zhu, J., Li, K., Chen, S., Fan, L., Wang, J., and Xie, X. (2024). A comprehensive study on static application security testing (sast) tools for android. IEEE Trans. Softw. Eng., 50(12):3385–3402.