Code readability assessment and improvement: human and automated perspectives
Resumo
Code readability is a fundamental characteristic of software quality. However, challenges remain in both its assessment and the automation of readability improvements. This article investigates code readability from complementary human and automated perspectives. First, it evaluates the readability of code generated by Large Language Models (LLMs) compared to solutions written by developers. Second, it characterizes how developers improve readability in GitHub repositories through the construction of a taxonomy of readability improvements. Finally, it proposes and evaluates a hybrid approach that combines static analysis tools and LLMs to automatically improve code readability. A unified taxonomy comprising 56 types of readability improvements observed in Java and Python projects was developed. The results also show that the code generated by LLMs frequently exhibits fewer readability issues than human-written examples. Additionally, the combination of static analysis recommendations and LLM-based refactorings proved effective in reducing readability issues. Together, these findings advance our understanding of code readability and contribute to the development of tools to improve it.
Palavras-chave:
code readability, large language models, pull request, code review, automatic static analysis tools
Referências
K.K. Aggarwal, Y. Singh, and J.K. Chhabra. 2002. An integrated measure of software maintainability. In Annual Reliability and Maintainability Symposium. 2002 Proceedings (Cat. No.02CH37318). 235–241. DOI: 10.1109/RAMS.2002.981648
Fabian Beck and Stephan Diehl. 2011. On the Congruence of Modularity and Code Coupling. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (Szeged, Hungary) (ESEC/FSE ’11). Association for Computing Machinery, New York, NY, USA, 354–364. DOI: 10.1145/2025113.2025162
Moritz Beller, Radjino Bholanath, Shane McIntosh, and Andy Zaidman. 2016. Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 1. 470–481. DOI: 10.1109/SANER.2016.105
Hans Christian Benestad, Bente Anda, and Erik Arisholm. 2009. Understanding software maintenance and evolution by analyzing individual changes: a literature review. J. Softw. Maint. Evol. 21, 6 (Nov. 2009), 349–378. DOI: 10.1002/smr.412
Keith H. Bennett and Václav T. Rajlich. 2000. Software maintenance and evolution: a roadmap. In Proceedings of the Conference on The Future of Software Engineering (Limerick, Ireland) (ICSE ’00). Association for Computing Machinery, New York, NY, USA, 73–87. DOI: 10.1145/336512.336534
Raymond P.L. Buse and Westley R. Weimer. 2010. Learning a Metric for Code Readability. IEEE Transactions on Software Engineering 36, 4 (2010), 546–558. DOI: 10.1109/TSE.2009.70
Felipe Calixto, Eliane Araújo, and Everton Alves. 2024. How does Technical Debt Evolve within Pull Requests? An Empirical Study with Apache Projects. In Anais do XXXVIII Simpósio Brasileiro de Engenharia de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 212–223. DOI: 10.5753/sbes.2024.3368
Rodrigo Fernandes Gomes da Silva, Chanchal K. Roy, Mohammad Masudur Rahman, Kevin A. Schneider, Klérisson V. R. Paixão, Carlos Eduardo de Carvalho Dantas, and Marcelo de Almeida Maia. 2020. CROKAGE: effective solution recommendation for programming tasks by leveraging crowd knowledge. Empir. Softw. Eng. 25, 6 (2020), 4707–4758. DOI: 10.1007/s10664-020-09863-2
Carlos Eduardo C. Dantas and Marcelo A. Maia. 2025. Code Readability Assessment and Improvement: Human and Automated Perspectives. [link].
Jonathan Dorn. 2012. A General Software Readability Model. [link]
Sarah Fakhoury, Devjeet Roy, Sk. Adnan Hassan, and Venera Arnaoudova. 2019. Improving Source Code Readability: Theory and Practice. In Proceedings of the 27th International Conference on Program Comprehension (Montreal, Quebec, Canada) (ICPC ’19). IEEE Press, 2–12. DOI: 10.1109/ICPC.2019.00014
Prashant Gosavi and Václav T. Rajlich. 2004. Incremental Change in Object-Oriented Programming . IEEE Software 21, 04 (July 2004), 62–69. DOI: 10.1109/MS.2004.17
Qi Guo, Junming Cao, Xiaofei Xie, Shangqing Liu, Xiaohong Li, Bihuan Chen, and Xin Peng. 2024. Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Study. In Proceedings of the IEEE/ACM46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24). Association for Computing Machinery, New York, NY, USA, Article 34, 13 pages. DOI: 10.1145/3597503.3623306
Nuzhat J. Haneef. 1998. Software documentation and readability: a proposed process improvement. SIGSOFT Softw. Eng. Notes 23, 3 (May 1998), 75–77. DOI: 10.1145/279437.279470
Andre Hora. 2021. Googling for Software Development: What Developers Search For and What They Find. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). 317–328. DOI: 10.1109/MSR52588.2021.00044
André C. Hora. 2021. APISonar: Mining API usage examples. Software: Practice and Experience 51 (2021), 319 – 352. DOI: 10.1002/spe.2906
Imen Jaoua, Oussama Ben Sghaier, and Houari Sahraoui. 2025. Combining Large Language Models with Static Analyzers for Code Review Generation . In 2025 IEEE/ACM 22nd International Conference on Mining Software Repositories (MSR). IEEE Computer Society, Los Alamitos, CA, USA, 174–186. DOI: 10.1109/MSR66628.2025.00038
Chris Kemerer. 1995. Software Complexity and Software Maintenance: A Survey of Empirical Research. Ann. Software Eng. 1 (12 1995), 1–22. DOI: 10.1007/BF02249043
Valentina Lenarduzzi, Fabiano Pecorelli, Nyyti Saarimaki, Savanna Lujan, and Fabio Palomba. 2023. A critical comparison on six static analysis tools: Detection, agreement, and precision. Journal of Systems and Software 198 (2023), 111575. DOI: 10.1016/j.jss.2022.111575
Benjamin Loriot, Fer Madeiral, and Martin Monperrus. 2022. Styler: learning formatting conventions to repair Checkstyle violations. Empirical Software Engineering 27 (08 2022). DOI: 10.1007/s10664-021-10107-0
Diego Marcilio, Rodrigo Bonifácio, Eduardo Monteiro, Edna Canedo, Welder Luz, and Gustavo Pinto. 2019. Are Static Analysis Violations Really Fixed? A Closer Look at Realistic Usage of SonarQube. In 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC). 209–219. DOI: 10.1109/ICPC.2019.00040
Roberto Minelli, Andrea Mocci and, and Michele Lanza. 2015. I knowwhat you did last summer: an investigation of how developers spend their time. In Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension (Florence, Italy) (ICPC ’15). IEEE Press, 25–35. DOI: 10.1109/ICPC.2015.12
Laura Moreno, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Andrian Marcus. 2015. How Can I Use This Method?. In Proceedings of the 37th International Conference on Software Engineering - Volume 1 (Florence, Italy) (ICSE ’15). IEEE Press, 880–890. DOI: 10.1109/ICSE.2015.98
Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an LLM to Help With Code Understanding. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24). Association for Computing Machinery, New York, NY, USA, Article 97, 13 pages. DOI: 10.1145/3597503.3639187
Delano Oliveira, Reydne Santos, Benedito de Oliveira, Martin Monperrus, Fernando Castor, and Fernanda Madeiral. 2025. Understanding Code Understandability Improvements in Code Reviews. IEEE Transactions on Software Engineering 51, 1 (2025), 14–37. DOI: 10.1109/TSE.2024.3453783
Jevgenija Pantiuchina, Michele Lanza, and Gabriele Bavota. 2018. Improving Code: The (Mis) Perception of Quality Metrics. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). 80–91. DOI: 10.1109/ICSME.2018.00017
Jevgenija Pantiuchina, Fiorella Zampetti, Simone Scalabrino, Valentina Piantadosi, Rocco Oliveto, Gabriele Bavota, and Massimiliano Di Penta. 2020. Why Developers Refactor Source Code: A Mining-based Study. ACM Trans. Softw. Eng. Methodol. 29, 4, Article 29 (Sept. 2020), 30 pages. DOI: 10.1145/3408302
Valentina Piantadosi, Fabiana Fierro, Simone Scalabrino, Alexander Serebrenik, and Rocco Oliveto. 2020. How does code readability change during software evolution? Empirical Software Engineering 25 (11 2020), 1–39. DOI: 10.1007/s10664-020-09886-9
Daryl Posnett, Abram Hindle, and Premkumar Devanbu. 2011. A simpler model of software readability. In Proceedings of the 8th Working Conference on Mining Software Repositories (Waikiki, Honolulu, HI, USA) (MSR ’11). Association for Computing Machinery, New York, NY, USA, 73–82. DOI: 10.1145/1985441.1985454
Simone Romano, Fiorella Zampetti, Maria Teresa Baldassarre, Massimiliano Di Penta, and Giuseppe Scanniello. 2022. Do Static Analysis Tools Affect Software Quality when Using Test-driven Development?. In Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (Helsinki, Finland) (ESEM ’22). Association for Computing Machinery, New York, NY, USA, 80–91. DOI: 10.1145/3544902.3546233
Devjeet Roy, Sarah Fakhoury, John Lee, and Venera Arnaoudova. 2020. A Model to Detect Readability Improvements in Incremental Changes. Association for Computing Machinery, New York, NY, USA, 25–36. DOI: 10.1145/3387904.3389255
Simone Scalabrino, Mario Linares-Vásquez, Denys Poshyvanyk, and Rocco Oliveto. 2016. Improving code readability models with textual features. In 2016 IEEE 24th International Conference on Program Comprehension (ICPC). 1–10. DOI: 10.1109/ICPC.2016.7503707
Simone Scalabrino, Mario Linares-Vásquez, Rocco Oliveto, and Denys Poshyvanyk. 2018. A comprehensive model for code readability. Journal of Software: Evolution and Process 30 (06 2018). DOI: 10.1002/smr.1958
Julyanara Silva, Carlos Dantas, and Marcelo Maia. 2024. What Developers Ask to ChatGPT in GitHub Pull Requests? an Exploratory Study. In Anais do XII Workshop de Visualização, Evolução e Manutenção de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 125–136. DOI: 10.5753/vem.2024.3913
SONARLINT. 2025. SonarlintWeb Page. [link]. Accessed: 2025-02-20.
Sean Stapleton, Yashmeet Gambhir, Alexander LeClair, Zachary Eberhart,Westley Weimer, Kevin Leach, and Yu Huang. 2020. A Human Study of Comprehension and Code Summarization. In Proceedings of the 28th International Conference on Program Comprehension (Seoul, Republic of Korea) (ICPC ’20). Association for Computing Machinery, New York, NY, USA, 2–13. DOI: 10.1145/3387904.3389258
Rebecca Tiarks. 2011. What Programmers Really Do - An Observational Study. Softwaretechnik-Trends 31 (2011). [link]
Rosalia Tufano, Antonio Mastropaolo, Federica Pepe, Ozren Dabic, Massimiliano Di Penta, and Gabriele Bavota. 2024. Unveiling ChatGPT’s Usage in Open Source Projects: A Mining-based Study. In Proceedings of the 21st International Conference on Mining Software Repositories (Lisbon, Portugal) (MSR ’24). Association for Computing Machinery, New York, NY, USA, 571–583. DOI: 10.1145/3643991.3644918
Tao Xiao, Christoph Treude, Hideaki Hata, and Kenichi Matsumoto. 2024. DevGPT: Studying Developer-ChatGPT Conversations. In Proceedings of the International Conference on Mining Software Repositories (MSR 2024). DOI: 10.1145/3643991.3648400
Fabian Beck and Stephan Diehl. 2011. On the Congruence of Modularity and Code Coupling. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (Szeged, Hungary) (ESEC/FSE ’11). Association for Computing Machinery, New York, NY, USA, 354–364. DOI: 10.1145/2025113.2025162
Moritz Beller, Radjino Bholanath, Shane McIntosh, and Andy Zaidman. 2016. Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 1. 470–481. DOI: 10.1109/SANER.2016.105
Hans Christian Benestad, Bente Anda, and Erik Arisholm. 2009. Understanding software maintenance and evolution by analyzing individual changes: a literature review. J. Softw. Maint. Evol. 21, 6 (Nov. 2009), 349–378. DOI: 10.1002/smr.412
Keith H. Bennett and Václav T. Rajlich. 2000. Software maintenance and evolution: a roadmap. In Proceedings of the Conference on The Future of Software Engineering (Limerick, Ireland) (ICSE ’00). Association for Computing Machinery, New York, NY, USA, 73–87. DOI: 10.1145/336512.336534
Raymond P.L. Buse and Westley R. Weimer. 2010. Learning a Metric for Code Readability. IEEE Transactions on Software Engineering 36, 4 (2010), 546–558. DOI: 10.1109/TSE.2009.70
Felipe Calixto, Eliane Araújo, and Everton Alves. 2024. How does Technical Debt Evolve within Pull Requests? An Empirical Study with Apache Projects. In Anais do XXXVIII Simpósio Brasileiro de Engenharia de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 212–223. DOI: 10.5753/sbes.2024.3368
Rodrigo Fernandes Gomes da Silva, Chanchal K. Roy, Mohammad Masudur Rahman, Kevin A. Schneider, Klérisson V. R. Paixão, Carlos Eduardo de Carvalho Dantas, and Marcelo de Almeida Maia. 2020. CROKAGE: effective solution recommendation for programming tasks by leveraging crowd knowledge. Empir. Softw. Eng. 25, 6 (2020), 4707–4758. DOI: 10.1007/s10664-020-09863-2
Carlos Eduardo C. Dantas and Marcelo A. Maia. 2025. Code Readability Assessment and Improvement: Human and Automated Perspectives. [link].
Jonathan Dorn. 2012. A General Software Readability Model. [link]
Sarah Fakhoury, Devjeet Roy, Sk. Adnan Hassan, and Venera Arnaoudova. 2019. Improving Source Code Readability: Theory and Practice. In Proceedings of the 27th International Conference on Program Comprehension (Montreal, Quebec, Canada) (ICPC ’19). IEEE Press, 2–12. DOI: 10.1109/ICPC.2019.00014
Prashant Gosavi and Václav T. Rajlich. 2004. Incremental Change in Object-Oriented Programming . IEEE Software 21, 04 (July 2004), 62–69. DOI: 10.1109/MS.2004.17
Qi Guo, Junming Cao, Xiaofei Xie, Shangqing Liu, Xiaohong Li, Bihuan Chen, and Xin Peng. 2024. Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Study. In Proceedings of the IEEE/ACM46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24). Association for Computing Machinery, New York, NY, USA, Article 34, 13 pages. DOI: 10.1145/3597503.3623306
Nuzhat J. Haneef. 1998. Software documentation and readability: a proposed process improvement. SIGSOFT Softw. Eng. Notes 23, 3 (May 1998), 75–77. DOI: 10.1145/279437.279470
Andre Hora. 2021. Googling for Software Development: What Developers Search For and What They Find. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). 317–328. DOI: 10.1109/MSR52588.2021.00044
André C. Hora. 2021. APISonar: Mining API usage examples. Software: Practice and Experience 51 (2021), 319 – 352. DOI: 10.1002/spe.2906
Imen Jaoua, Oussama Ben Sghaier, and Houari Sahraoui. 2025. Combining Large Language Models with Static Analyzers for Code Review Generation . In 2025 IEEE/ACM 22nd International Conference on Mining Software Repositories (MSR). IEEE Computer Society, Los Alamitos, CA, USA, 174–186. DOI: 10.1109/MSR66628.2025.00038
Chris Kemerer. 1995. Software Complexity and Software Maintenance: A Survey of Empirical Research. Ann. Software Eng. 1 (12 1995), 1–22. DOI: 10.1007/BF02249043
Valentina Lenarduzzi, Fabiano Pecorelli, Nyyti Saarimaki, Savanna Lujan, and Fabio Palomba. 2023. A critical comparison on six static analysis tools: Detection, agreement, and precision. Journal of Systems and Software 198 (2023), 111575. DOI: 10.1016/j.jss.2022.111575
Benjamin Loriot, Fer Madeiral, and Martin Monperrus. 2022. Styler: learning formatting conventions to repair Checkstyle violations. Empirical Software Engineering 27 (08 2022). DOI: 10.1007/s10664-021-10107-0
Diego Marcilio, Rodrigo Bonifácio, Eduardo Monteiro, Edna Canedo, Welder Luz, and Gustavo Pinto. 2019. Are Static Analysis Violations Really Fixed? A Closer Look at Realistic Usage of SonarQube. In 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC). 209–219. DOI: 10.1109/ICPC.2019.00040
Roberto Minelli, Andrea Mocci and, and Michele Lanza. 2015. I knowwhat you did last summer: an investigation of how developers spend their time. In Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension (Florence, Italy) (ICPC ’15). IEEE Press, 25–35. DOI: 10.1109/ICPC.2015.12
Laura Moreno, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Andrian Marcus. 2015. How Can I Use This Method?. In Proceedings of the 37th International Conference on Software Engineering - Volume 1 (Florence, Italy) (ICSE ’15). IEEE Press, 880–890. DOI: 10.1109/ICSE.2015.98
Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an LLM to Help With Code Understanding. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24). Association for Computing Machinery, New York, NY, USA, Article 97, 13 pages. DOI: 10.1145/3597503.3639187
Delano Oliveira, Reydne Santos, Benedito de Oliveira, Martin Monperrus, Fernando Castor, and Fernanda Madeiral. 2025. Understanding Code Understandability Improvements in Code Reviews. IEEE Transactions on Software Engineering 51, 1 (2025), 14–37. DOI: 10.1109/TSE.2024.3453783
Jevgenija Pantiuchina, Michele Lanza, and Gabriele Bavota. 2018. Improving Code: The (Mis) Perception of Quality Metrics. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). 80–91. DOI: 10.1109/ICSME.2018.00017
Jevgenija Pantiuchina, Fiorella Zampetti, Simone Scalabrino, Valentina Piantadosi, Rocco Oliveto, Gabriele Bavota, and Massimiliano Di Penta. 2020. Why Developers Refactor Source Code: A Mining-based Study. ACM Trans. Softw. Eng. Methodol. 29, 4, Article 29 (Sept. 2020), 30 pages. DOI: 10.1145/3408302
Valentina Piantadosi, Fabiana Fierro, Simone Scalabrino, Alexander Serebrenik, and Rocco Oliveto. 2020. How does code readability change during software evolution? Empirical Software Engineering 25 (11 2020), 1–39. DOI: 10.1007/s10664-020-09886-9
Daryl Posnett, Abram Hindle, and Premkumar Devanbu. 2011. A simpler model of software readability. In Proceedings of the 8th Working Conference on Mining Software Repositories (Waikiki, Honolulu, HI, USA) (MSR ’11). Association for Computing Machinery, New York, NY, USA, 73–82. DOI: 10.1145/1985441.1985454
Simone Romano, Fiorella Zampetti, Maria Teresa Baldassarre, Massimiliano Di Penta, and Giuseppe Scanniello. 2022. Do Static Analysis Tools Affect Software Quality when Using Test-driven Development?. In Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (Helsinki, Finland) (ESEM ’22). Association for Computing Machinery, New York, NY, USA, 80–91. DOI: 10.1145/3544902.3546233
Devjeet Roy, Sarah Fakhoury, John Lee, and Venera Arnaoudova. 2020. A Model to Detect Readability Improvements in Incremental Changes. Association for Computing Machinery, New York, NY, USA, 25–36. DOI: 10.1145/3387904.3389255
Simone Scalabrino, Mario Linares-Vásquez, Denys Poshyvanyk, and Rocco Oliveto. 2016. Improving code readability models with textual features. In 2016 IEEE 24th International Conference on Program Comprehension (ICPC). 1–10. DOI: 10.1109/ICPC.2016.7503707
Simone Scalabrino, Mario Linares-Vásquez, Rocco Oliveto, and Denys Poshyvanyk. 2018. A comprehensive model for code readability. Journal of Software: Evolution and Process 30 (06 2018). DOI: 10.1002/smr.1958
Julyanara Silva, Carlos Dantas, and Marcelo Maia. 2024. What Developers Ask to ChatGPT in GitHub Pull Requests? an Exploratory Study. In Anais do XII Workshop de Visualização, Evolução e Manutenção de Software (Curitiba/PR). SBC, Porto Alegre, RS, Brasil, 125–136. DOI: 10.5753/vem.2024.3913
SONARLINT. 2025. SonarlintWeb Page. [link]. Accessed: 2025-02-20.
Sean Stapleton, Yashmeet Gambhir, Alexander LeClair, Zachary Eberhart,Westley Weimer, Kevin Leach, and Yu Huang. 2020. A Human Study of Comprehension and Code Summarization. In Proceedings of the 28th International Conference on Program Comprehension (Seoul, Republic of Korea) (ICPC ’20). Association for Computing Machinery, New York, NY, USA, 2–13. DOI: 10.1145/3387904.3389258
Rebecca Tiarks. 2011. What Programmers Really Do - An Observational Study. Softwaretechnik-Trends 31 (2011). [link]
Rosalia Tufano, Antonio Mastropaolo, Federica Pepe, Ozren Dabic, Massimiliano Di Penta, and Gabriele Bavota. 2024. Unveiling ChatGPT’s Usage in Open Source Projects: A Mining-based Study. In Proceedings of the 21st International Conference on Mining Software Repositories (Lisbon, Portugal) (MSR ’24). Association for Computing Machinery, New York, NY, USA, 571–583. DOI: 10.1145/3643991.3644918
Tao Xiao, Christoph Treude, Hideaki Hata, and Kenichi Matsumoto. 2024. DevGPT: Studying Developer-ChatGPT Conversations. In Proceedings of the International Conference on Mining Software Repositories (MSR 2024). DOI: 10.1145/3643991.3648400
Publicado
08/09/2026
Como Citar
DANTAS, Carlos Eduardo de Carvalho; MAIA, Marcelo de Almeida.
Code readability assessment and improvement: human and automated perspectives. In: CONCURSO DE TESES E DISSERTAÇÕES EM ENGENHARIA DE SOFTWARE (DOUTORADO) - CONGRESSO BRASILEIRO DE SOFTWARE: TEORIA E PRÁTICA (CBSOFT), 17. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 14-22.
DOI: https://doi.org/10.5753/cbsoft_estendido.2026.29700.
