Eyes on Code Smells: Analyzing Developers’ Responses During Code Snippet Analysis
Resumo
Detection and refactoring of smelly code are crucial activities, along with software maintenance and evolution. Code smells are indicators of poor design and implementation choices, which are expected to affect the developers’ program comprehension. The use of eyetracking technology provides an interesting means of analyzing the impact of code smells on program comprehension. However, there is limited existing effort in this direction. This paper reports a study in which we have used an eye tracker to investigate how the presence of smells influences developers’ program comprehension. We observed that the smell data class leads to a lower cognitive effort, while long methods and feature envies imposed a considerably higher cognitive effort. That explains why refactoring of feature envies and long methods smells has been much more common across the projects.We also have complementary eye-tracking indicators to reveal other smell aspects harming program comprehension. These findings reinforce the importance of enhancing IDE features to reduce the developer’s burden when engaging in cognitive processes of largely coupled code.
Palavras-chave:
eye tracker, code smell, software quality, program comprehension
Referências
[n. d.]. Tobii TX300 Eye Tracker. [link]. Accessed: January 17, 2024.
Juan Pablo Sandoval Alcocer, Alejandra Cossio-Chavalier, Tiara Rojas-Stambuk, and Leonel Merino. 2023. An Eye-Tracking Study on the Use of Split/Unified Code Change Views for Bug Detection. IEEE Access 11 (2023), 136195–136205. DOI: 10.1109/ACCESS.2023.3336859
F. Arcelli Fontana, M.V. Mäntylä, M. Zanoni, et al. 2016. Comparing and experimenting machine learning techniques for code smell detection. Empirical Software Engineering 21 (2016), 1143–1191. DOI: 10.1007/s10664-015-9378-4
Francesca Arcelli Fontana and Marco Zanoni. 2017. Code smell severity classification using machine learning techniques. Knowledge-Based Systems 128 (2017), 43–58. Università degli Studi di Milano-Bicocca, Milan, Italy.
R Bednarik. 2012. Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations. International Journal of Human-Computer Studies 70, 2 (2012), 143–155. DOI: 10.1016/j.ijhcs.2011.09.003
Joshua Behler, Praxis Weston, Drew T. Guarnera, Bonita Sharif, and Jonathan I. Maletic. 2023. iTrace-Toolkit: A Pipeline for Analyzing Eye-Tracking Data of Software Engineering Studies. In 2023 IEEE/ACM 45th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). 46–50. DOI: 10.1109/ICSE-Companion58688.2023.00022
Ana Carla Bibiano, Anderson Uchôa, Wesley K.G. Assunção, Daniel Tenório, Thelma E. Colanzi, Silvia Regina Vergilio, and Alessandro Garcia. 2023. Composite refactoring: Representations, characteristics and effects on software projects. Information and Software Technology 156 (2023), 107134. DOI: 10.1016/j.infsof.2022.107134
D Binkley,MDavis, D Lawrie, JI Maletic, C Morrell, and B Sharif. 2013. The impact of identifier style on effort and comprehension. Empirical Software Engineering 18, 2 (2013), 219–276. DOI: 10.1007/s10664-012-9201-4
William J. Brown. 1998. AntiPatterns: refactoring software architectures and projects in crisis. Wiley, New York.
Diego Cedrim, Alessandro Garcia, Melina Mongiovi, Rohit Gheyi, Leonardo Sousa, Rafael de Mello, Baldoino Fonseca, Márcio Ribeiro, and Alexander Chávez. 2017. Understanding the impact of refactoring on smells: a longitudinal study of 23 software projects. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 465–475. DOI: 10.1145/3106237.3106259
Martha Crosby, Jean Scholtz, and Susan Wiedenbeck. 2002. The roles beacons play in comprehension for novice and expert programmers. (07 2002).
Dipta Das, Abdullah Al Maruf, Rofiqul Islam, Noah Lambaria, Samuel Kim, Amr S. Abdelfattah, Tomas Cerny, Karel Frajtak, Miroslav Bures, and Pavel Tisnovsky. 2022. Technical debt resulting from architectural degradation and code smells: a systematic mapping study. SIGAPP Appl. Comput. Rev. 21, 4 (jan 2022), 20–36. DOI: 10.1145/3512753.3512755
Daniel Kyle Davis and Feng Zhu. 2022. Analysis of software developers’ coding behavior: A survey of visualization analysis techniques using eye trackers. Computers in Human Behavior Reports 7 (2022), 100213.
Martin Fowler and Kent Beck. 1999. Refactoring: Improving the Design of Existing Code. Addison-Wesley Longman Publishing Co., Inc., USA.
Yu Huang, Kevin Leach, Zohreh Sharafi, Nicholas McKay, Tyler Santander, and Westley Weimer. 2020. Biases and differences in code review using medical imaging and eye-tracking: genders, humans, and machines. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 456–468. DOI: 10.1145/3368089.3409681
Marcel A. Just and Patricia A. Carpenter. 1980. A theory of reading: From eye fixations to comprehension. Psychological Review 87, 4 (1980), 329–354. DOI: 10.1037/0033-295X.87.4.329
Guilherme Lacerda, Fabio Petrillo, Marcelo Pimenta, and Yann Gaël Guéhéneuc. 2020. Code smells and refactoring: A tertiary systematic review of challenges and observations. Journal of Systems and Software 167 (2020), 110610. DOI: 10.1016/j.jss.2020.110610
Lech Madeyski and Tomasz Lewowski. 2020. MLCQ: Industry-Relevant Code Smell Data Set. In Proceedings of the Evaluation and Assessment in Software Engineering (EASE ’20). Association for Computing Machinery, New York, NY, USA, 342–347. DOI: 10.1145/3383219.3383264
Mika Mäntylä. 2003. Bad Smells in Software - A Taxonomy and an Empirical Study. Ph.D. Dissertation. Helsinki University of Technology.
Mika V. Mäntylä and Casper Lassenius. 2006. Subjective evaluation of software evolvability using code smells: An empirical study. Empirical Software Engineering 11 (2006), 395–431. DOI: 10.1007/s10664-006-9002-8
Vinícius Martins, Pedro Lopes Verardo Ramos, Breno Braga Neves, Maria Vitoria Lima, Johny Arriel, João Victor Godinho, Joanne Ribeiro, Alessandro Garcia, and Juliana Alves Pereira. 2024. Eyes on Code Smells: Analyzing Developers’ Responses During Code Snippet Analysis. [link]. Accessed: 2024-07-25.
Daniel Oliveira, Wesley K. G. Assunção, Alessandro Garcia, Ana Carla Bibiano, Márcio Ribeiro, Rohit Gheyi, and Baldoino Fonseca. 2023. The untold story of code refactoring customizations in practice. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 108–120. DOI: 10.1109/ICSE48619.2023.00021
Matheus Paixão, Anderson Uchôa, Ana Carla Bibiano, Daniel Oliveira, Alessandro Garcia, Jens Krinke, and Emilio Arvonio. 2020. Behind the Intents: An In-depth Empirical Study on Software Refactoring in Modern Code Review. In Proceedings of the 17th International Conference on Mining Software Repositories (Seoul, Republic of Korea) (MSR ’20). Association for Computing Machinery, New York, NY, USA, 125–136. DOI: 10.1145/3379597.3387475
Fabio Palomba, Dario Di Nucci, Michele Tufano, Gabriele Bavota, Rocco Oliveto, Denys Poshyvanyk, and Andrea De Lucia. 2015. Landfill: An Open Dataset of Code Smells with Public Evaluation. In 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories. 482–485. DOI: 10.1109/MSR.2015.69
Charlie Parker, Sam Scott, and Alistair Geddes. 2019. Snowball Sampling. SAGE Publications, Inc., London. DOI: 10.4135/9781526421036831710 Accessed on January 16, 2024.
Joseph R Pauszek. 2023. An introduction to eye tracking in human factors healthcare research and medical device testing. Human Factors in Healthcare 3 (2023), 100031.
Norman Peitek, Annabelle Bergum, Maurice Rekrut, Jonas Mucke, Matthias Nadig, Chris Parnin, Janet Siegmund, and Sven Apel. 2022. Correlates of programmer efficacy and their link to experience: a combined EEG and eye-tracking study. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Singapore) (ESEC/FSE 2022). Association for Computing Machinery, New York, NY, USA, 120–131. DOI: 10.1145/3540250.3549084
K. Rayner. 1998. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 124, 3 (1998), 372–422. DOI: 10.1037/0033-2909.124.3.372
Arthur J. Riel. 1996. Object-oriented design heuristics. Addison-Wesley Pub. Co, Reading, Mass.
Timothy R. Shaffer, Jenna L. Wise, Braden M. Walters, Sebastian C. Müller, Michael Falcone, and Bonita Sharif. 2015. ITrace: Enabling Eye Tracking on Software Artifacts within the IDE to Support Software Engineering Tasks. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE 2015). Association for Computing Machinery, New York, NY, USA, 954–957. DOI: 10.1145/2786805.2803188
Z Sharafi, T Shaffer, S Bonita, and YG Guéhéneuc. 2015. Eye-tracking metrics in software engineering. In Proceedings of the 22nd Asia-Pacific Software Engineering Conference (APSEC ’15). IEEE CS Press.
Zohreh Sharafi, Bonita Sharif, Yann-Gaël Guéhéneuc, Andrew Begel, Roman Bednarik, and Martha Crosby. 2020. A Practical Guide on Conducting Eye Tracking Studies in Software Engineering. Empirical Softw. Engg. 25, 5 (sep 2020), 3128–3174. DOI: 10.1007/s10664-020-09829-4
Zohreh Sharafi, Zéphyrin Soh, and Yann-Gaël Guéhéneuc. 2015. A systematic literature review on the usage of eye-tracking in software engineering. Information and Software Technology 67 (2015), 79–107. DOI: 10.1016/j.infsof.2015.06.008
B Sharif, M Falcone, and JI Maletic. 2012. An eye-tracking study on the role of scan time in finding source code defects. In Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA’12). ACM, New York, 381–384.
Juan Pablo Sandoval Alcocer, Alejandra Cossio-Chavalier, Tiara Rojas-Stambuk, and Leonel Merino. 2023. An Eye-Tracking Study on the Use of Split/Unified Code Change Views for Bug Detection. IEEE Access 11 (2023), 136195–136205. DOI: 10.1109/ACCESS.2023.3336859
F. Arcelli Fontana, M.V. Mäntylä, M. Zanoni, et al. 2016. Comparing and experimenting machine learning techniques for code smell detection. Empirical Software Engineering 21 (2016), 1143–1191. DOI: 10.1007/s10664-015-9378-4
Francesca Arcelli Fontana and Marco Zanoni. 2017. Code smell severity classification using machine learning techniques. Knowledge-Based Systems 128 (2017), 43–58. Università degli Studi di Milano-Bicocca, Milan, Italy.
R Bednarik. 2012. Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations. International Journal of Human-Computer Studies 70, 2 (2012), 143–155. DOI: 10.1016/j.ijhcs.2011.09.003
Joshua Behler, Praxis Weston, Drew T. Guarnera, Bonita Sharif, and Jonathan I. Maletic. 2023. iTrace-Toolkit: A Pipeline for Analyzing Eye-Tracking Data of Software Engineering Studies. In 2023 IEEE/ACM 45th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). 46–50. DOI: 10.1109/ICSE-Companion58688.2023.00022
Ana Carla Bibiano, Anderson Uchôa, Wesley K.G. Assunção, Daniel Tenório, Thelma E. Colanzi, Silvia Regina Vergilio, and Alessandro Garcia. 2023. Composite refactoring: Representations, characteristics and effects on software projects. Information and Software Technology 156 (2023), 107134. DOI: 10.1016/j.infsof.2022.107134
D Binkley,MDavis, D Lawrie, JI Maletic, C Morrell, and B Sharif. 2013. The impact of identifier style on effort and comprehension. Empirical Software Engineering 18, 2 (2013), 219–276. DOI: 10.1007/s10664-012-9201-4
William J. Brown. 1998. AntiPatterns: refactoring software architectures and projects in crisis. Wiley, New York.
Diego Cedrim, Alessandro Garcia, Melina Mongiovi, Rohit Gheyi, Leonardo Sousa, Rafael de Mello, Baldoino Fonseca, Márcio Ribeiro, and Alexander Chávez. 2017. Understanding the impact of refactoring on smells: a longitudinal study of 23 software projects. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 465–475. DOI: 10.1145/3106237.3106259
Martha Crosby, Jean Scholtz, and Susan Wiedenbeck. 2002. The roles beacons play in comprehension for novice and expert programmers. (07 2002).
Dipta Das, Abdullah Al Maruf, Rofiqul Islam, Noah Lambaria, Samuel Kim, Amr S. Abdelfattah, Tomas Cerny, Karel Frajtak, Miroslav Bures, and Pavel Tisnovsky. 2022. Technical debt resulting from architectural degradation and code smells: a systematic mapping study. SIGAPP Appl. Comput. Rev. 21, 4 (jan 2022), 20–36. DOI: 10.1145/3512753.3512755
Daniel Kyle Davis and Feng Zhu. 2022. Analysis of software developers’ coding behavior: A survey of visualization analysis techniques using eye trackers. Computers in Human Behavior Reports 7 (2022), 100213.
Martin Fowler and Kent Beck. 1999. Refactoring: Improving the Design of Existing Code. Addison-Wesley Longman Publishing Co., Inc., USA.
Yu Huang, Kevin Leach, Zohreh Sharafi, Nicholas McKay, Tyler Santander, and Westley Weimer. 2020. Biases and differences in code review using medical imaging and eye-tracking: genders, humans, and machines. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 456–468. DOI: 10.1145/3368089.3409681
Marcel A. Just and Patricia A. Carpenter. 1980. A theory of reading: From eye fixations to comprehension. Psychological Review 87, 4 (1980), 329–354. DOI: 10.1037/0033-295X.87.4.329
Guilherme Lacerda, Fabio Petrillo, Marcelo Pimenta, and Yann Gaël Guéhéneuc. 2020. Code smells and refactoring: A tertiary systematic review of challenges and observations. Journal of Systems and Software 167 (2020), 110610. DOI: 10.1016/j.jss.2020.110610
Lech Madeyski and Tomasz Lewowski. 2020. MLCQ: Industry-Relevant Code Smell Data Set. In Proceedings of the Evaluation and Assessment in Software Engineering (EASE ’20). Association for Computing Machinery, New York, NY, USA, 342–347. DOI: 10.1145/3383219.3383264
Mika Mäntylä. 2003. Bad Smells in Software - A Taxonomy and an Empirical Study. Ph.D. Dissertation. Helsinki University of Technology.
Mika V. Mäntylä and Casper Lassenius. 2006. Subjective evaluation of software evolvability using code smells: An empirical study. Empirical Software Engineering 11 (2006), 395–431. DOI: 10.1007/s10664-006-9002-8
Vinícius Martins, Pedro Lopes Verardo Ramos, Breno Braga Neves, Maria Vitoria Lima, Johny Arriel, João Victor Godinho, Joanne Ribeiro, Alessandro Garcia, and Juliana Alves Pereira. 2024. Eyes on Code Smells: Analyzing Developers’ Responses During Code Snippet Analysis. [link]. Accessed: 2024-07-25.
Daniel Oliveira, Wesley K. G. Assunção, Alessandro Garcia, Ana Carla Bibiano, Márcio Ribeiro, Rohit Gheyi, and Baldoino Fonseca. 2023. The untold story of code refactoring customizations in practice. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 108–120. DOI: 10.1109/ICSE48619.2023.00021
Matheus Paixão, Anderson Uchôa, Ana Carla Bibiano, Daniel Oliveira, Alessandro Garcia, Jens Krinke, and Emilio Arvonio. 2020. Behind the Intents: An In-depth Empirical Study on Software Refactoring in Modern Code Review. In Proceedings of the 17th International Conference on Mining Software Repositories (Seoul, Republic of Korea) (MSR ’20). Association for Computing Machinery, New York, NY, USA, 125–136. DOI: 10.1145/3379597.3387475
Fabio Palomba, Dario Di Nucci, Michele Tufano, Gabriele Bavota, Rocco Oliveto, Denys Poshyvanyk, and Andrea De Lucia. 2015. Landfill: An Open Dataset of Code Smells with Public Evaluation. In 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories. 482–485. DOI: 10.1109/MSR.2015.69
Charlie Parker, Sam Scott, and Alistair Geddes. 2019. Snowball Sampling. SAGE Publications, Inc., London. DOI: 10.4135/9781526421036831710 Accessed on January 16, 2024.
Joseph R Pauszek. 2023. An introduction to eye tracking in human factors healthcare research and medical device testing. Human Factors in Healthcare 3 (2023), 100031.
Norman Peitek, Annabelle Bergum, Maurice Rekrut, Jonas Mucke, Matthias Nadig, Chris Parnin, Janet Siegmund, and Sven Apel. 2022. Correlates of programmer efficacy and their link to experience: a combined EEG and eye-tracking study. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Singapore) (ESEC/FSE 2022). Association for Computing Machinery, New York, NY, USA, 120–131. DOI: 10.1145/3540250.3549084
K. Rayner. 1998. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 124, 3 (1998), 372–422. DOI: 10.1037/0033-2909.124.3.372
Arthur J. Riel. 1996. Object-oriented design heuristics. Addison-Wesley Pub. Co, Reading, Mass.
Timothy R. Shaffer, Jenna L. Wise, Braden M. Walters, Sebastian C. Müller, Michael Falcone, and Bonita Sharif. 2015. ITrace: Enabling Eye Tracking on Software Artifacts within the IDE to Support Software Engineering Tasks. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE 2015). Association for Computing Machinery, New York, NY, USA, 954–957. DOI: 10.1145/2786805.2803188
Z Sharafi, T Shaffer, S Bonita, and YG Guéhéneuc. 2015. Eye-tracking metrics in software engineering. In Proceedings of the 22nd Asia-Pacific Software Engineering Conference (APSEC ’15). IEEE CS Press.
Zohreh Sharafi, Bonita Sharif, Yann-Gaël Guéhéneuc, Andrew Begel, Roman Bednarik, and Martha Crosby. 2020. A Practical Guide on Conducting Eye Tracking Studies in Software Engineering. Empirical Softw. Engg. 25, 5 (sep 2020), 3128–3174. DOI: 10.1007/s10664-020-09829-4
Zohreh Sharafi, Zéphyrin Soh, and Yann-Gaël Guéhéneuc. 2015. A systematic literature review on the usage of eye-tracking in software engineering. Information and Software Technology 67 (2015), 79–107. DOI: 10.1016/j.infsof.2015.06.008
B Sharif, M Falcone, and JI Maletic. 2012. An eye-tracking study on the role of scan time in finding source code defects. In Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA’12). ACM, New York, 381–384.
Publicado
30/09/2024
Como Citar
MARTINS, Vinícius et al.
Eyes on Code Smells: Analyzing Developers’ Responses During Code Snippet Analysis. In: SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE (SBES), 38. , 2024, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 302-312.
DOI: https://doi.org/10.5753/sbes.2024.3431.
