PerfeQ: A Scientific Environment for Code Quality Analysis

• Francisco T. S. S. Pereira UEFS
• Roberto A. Bittencourt UEFS / University of Victoria
• Elaine H. T. Oliveira UFAM
• David B. F. Oliveira UFAM

DOI: https://doi.org/10.5753/educomp.2025.4937

Abstract

Faced with the need to improve the quality of students’ source code, researchers seek alternatives to provide feedback on the code not only regarding its correctness, but also its quality. However, large classrooms do not allow for individual and complete feedback. In this context, tools such as Static Analyzers (SAs) can be used to analyze code without executing it, providing reports on its quality, such as style issues. This paper presents a tool to evaluate code quality regarding the use of programming language style conventions. To this end, we created metrics to summarize the quantification of code quality based on the results of the SAs. Finally, we created an environment to simplify the scientific analysis of student code datasets. The work presents a detailed description of the environment, as well as illustrative examples of its operation.

References

Allamanis, M., Barr, E. T., Bird, C., and Sutton, C. (2014). Learning natural coding conventions. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2014, page 281–293, New York, NY, USA. Association for Computing Machinery.

Berry, R. E. and Meekings, B. A. (1985). A style analysis of c programs. Commun. ACM, 28(1):80–88.

Cannon, L., Elliott, R., Kirchhoff, L., Miller, J., Milner, J., Mitze, R., Schan, E., Whittington, N., Spencer, H., Keppel, D., et al. (1991). Recommended C style and coding standards. Pocket reference guide. Specialized Systems Consultants.

Doland, J. and Valett, J. (1994). C style guide. Technical report, National Aeronautics and Space Administration.

Dos Santos, G. T. H. and Martimiano, L. A. F. (2023). Uso de ferramentas de análise estática para identificar vulnerabilidades em sistemas operacionais em c/c++ para dispositivos iot. Revista Eletrônica de Iniciação Científica em Computação, 21(1).

dos Santos, R. M. and Gerosa, M. A. (2018). Impacts of coding practices on readability. In Proceedings of the 26th conference on program comprehension, pages 277–285.

Edwards, S. H., Kandru, N., and Rajagopal, M. B. (2017). Investigating static analysis errors in student java programs. In Proceedings of the 2017 ACM Conference on International Computing Education Research, ICER ’17, page 65–73, New York, NY, USA. ACM.

Finkbine, R. B. (1996). Metrics and Models in Software Quality Engineering, volume 21. Association for Computing Machinery, New York, NY, USA.

Galvão, L., Fernandes, D., and Gadelha, B. (2016). Juiz online como ferramenta de apoio a uma metodologia de ensino híbrido em programação. In (Simpósio Brasileiro de Informática na Educação - SBIE), volume 27, page 140.

Gomes, I., Morgado, P., Gomes, T., and Moreira, R. (2009). An overview on the static code analysis approach in software development. Faculdade de Engenharia da Universidade do Porto, Portugal.

Gulabovska, H. and Porkoláb, Z. (2019). Survey on static analysis tools of python programs. In SQAMIA.

Hedberg, H., Iivari, N., Rajanen, M., and Harjumaa, L. (2007). Assuring quality and usability in open soruce software development. In Proceedings of the First International Workshop on Emerging Trends in FLOSS Research and Development, FLOSS ’07, page 2, USA. IEEE Computer Society.

Joshi, A., Tewari, A., Kumar, V., and Bordoloi, D. (2014). Integrating static analysis tools for improving operating system security. International Journal of Computer Science and Mobile Computing, 3(4):1251–1258.

Keuning, H., Heeren, B., and Jeuring, J. (2017). Code quality issues in student programs. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE ’17. Association for Computing Machinery.

Liawatimena, S., Warnars, H. L. H. S., Trisetyarso, A., Abdurahman, E., Soewito, B., Wibowo, A., Gaol, F. L., and Abbas, B. S. (2018). Django web framework software metrics measurement using radon and pylint. In 2018 Indonesian Association for Pattern Recognition International Conference (INAPR), pages 218–222. IEEE.

Lima, Y., Fonseca, I., Chagas, J., Rodrigues, E., Silveira, M., and Silva, J. (2021). Comparação de ferramentas de análise estática para detecção de defeitos de software usando mutantes. In Anais da V Escola Regional de Engenharia de Software, pages 159–168, Porto Alegre, RS, Brasil. SBC.

Liu, D. and Petersen, A. (2019). Static analyses in python programming courses. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education, SIGCSE ’19, page 666–671, New York, NY, USA. Association for Computing Machinery.

Mengel, S. A. and Yerramilli, V. (1999). A case study of the static analysis of the quality of novice student programs. In The Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science Education, SIGCSE ’99, page 78–82, New York, NY, USA. Association for Computing Machinery.

Novak, J., Krajnc, A., et al. (2010). Taxonomy of static code analysis tools. In The 33rd international convention MIPRO, pages 418–422. IEEE.

Reiss, S. P. (2007). Automatic code stylizing. In Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering, ASE ’07, page 74–83, New York, NY, USA. Association for Computing Machinery.

Saliba, L., Shioji, E., Oliveira, E., Cohney, S., and Qi, J. (2024). Learning with style: Improving student code-style through better automated feedback. In Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1.

Van Rossum, G., Warsaw, B., and Coghlan, N. (2001). Pep 8–style guide for python code. Python. org, 1565:28.

Vorobyov, K. and Krishnan, P. (2010). Comparing model checking and static program analysis: A case study in error detection approaches. Proc. SSV, pages 1–7.

Weinberger, B., Silverstein, C., Eitzmann, G., Mentovai, M., and Landray, T. (2013). Google c++ style guide.