Code smell severity classification at class and method level with a single manually labeled imbalanced dataset

Resumo


Detecting code smells through machine learning (ML) poses challenges due to its unbalanced nature and potential interpretation bias. While previous studies focused on severity tended to categorize code smell’s specific types, this research aims to detect and classify code smell severity in a single dataset containing instances of code smells of four distinct types: God-class, Data-Class, Feature-Envy, and Long-Method. This study also explores the impact of applying data scaling, feature selection techniques, and ensemble methods to enhance ML models for the purpose above. The evaluation of two ensemble models on a combined dataset reveals that using data standardization techniques, ensemble methods, and Chi-square outperforms the result of other ensemble combinations, achieving 81.04% and 81.41% accuracy in the XGBoost and CatBoost models. Additionally, the CatBoost algorithm attains the highest accuracy at 80.67%, even without data preprocessing. Comparatively with the state-of-the-art, the results obtained, an accuracy of 85%, by the proposed approach in detecting the severity of code smells are promising and suggest improvements in approaches and techniques to enhance the effectiveness and reliability of models in real-world scenarios.

Palavras-chave: code smells, severity, data preprocessing, feature selection, ensemble methods, machine learning

Referências

Ashraf Abdou and Nagy Darwish. 2024. Severity classification of software code smells using machine learning techniques: A comparative study. Journal of Software: Evolution and Process 36, 1 (2024), e2454. DOI: 10.1002/smr. 2454 arXiv: [link]

Amal Alazba and Hamoud Aljamaan. 2021. Code smell detection using feature selection and stacking ensemble: An empirical investigation. Information and Software Technology 138 (2021), 106648. DOI: 10.1016/j.infsof.2021.106648

Khalid Alkharabsheh, Sadi Alawadi, Victor R. Kebande, Yania Crespo, Manuel Fernández-Delgado, and José A. Taboada. 2022. Acomparison of machine learning algorithms on design smell detection using balanced and imbalanced dataset: A study of God class. Information and Software Technology 143 (2022), 106736. DOI: 10.1016/j.infsof.2021.106736

Francesca Arcelli Fontana and Marco Zanoni. 2017. Code smell severity classification using machine learning techniques. Knowledge-Based Systems 128 (2017), 43–58. DOI: 10.1016/j.knosys.2017.04.014

Francesca Arcelli Fontana, Mika V. Mäntylä, Marco Zanoni, and Alessandro Marino. 2016. Comparing and Experimenting Machine Learning Techniques for Code Smell Detection. Empirical Softw. Engg. 21, 3 (jun 2016), 1143–1191. DOI: 10.1007/s10664-015-9378-4

Muhammad Ilyas Azeem, Fabio Palomba, Lin Shi, and QingWang. 2019. Machine learning techniques for code smell detection: A systematic literature review and meta-analysis. Information and Software Technology 108 (2019), 115–138. DOI: 10.1016/j.infsof.2018.12.009

L.C. Briand, J.W. Daly, and J.K.Wust. 1999. A unified framework for coupling measurement in object-oriented systems. IEEE Transactions on Software Engineering 25, 1 (1999), 91–121. DOI: 10.1109/32.748920

William H. Brown, Raphael C. Malveau, HaysW. "Skip" McCormick, and Thomas J. Mowbray. 1998. AntiPatterns: Refactoring Software, Architectures, and Projects in Crisis (1st ed.). John Wiley & Sons, Inc., USA.

Cagatay Catal. 2012. Performance Evaluation Metrics for Software Fault Prediction Studies. Acta Polytechnica Hungarica 9 (01 2012).

O. Ciupke. 1999. Automatic detection of design problems in object-oriented reengineering. In Proceedings of Technology of Object-Oriented Languages and Systems - TOOLS 30 (Cat. No.PR00278). IEEE, Santa Barbara, CA, USA, 18–32. DOI: 10.1109/TOOLS.1999.787532

Seema Dewangan, Rajwant Singh Rao, Sripriya Roy Chowdhuri, and Manjari Gupta. 2023. Severity Classification of Code Smells Using Machine-Learning Methods. SN Computer Science 4, 5 (2023). DOI: 10.1007/s42979-023-01979-8

Seema Dewangan, Rajwant Singh Rao, Alok Mishra, and Manjari Gupta. 2021. A novel approach for code smell detection: An empirical study. IEEE Access 9 (2021), 162869–162883. DOI: 10.1109/ACCESS.2021.3133810

Seema Dewangan, Rajwant Singh Rao, Alok Mishra, and Manjari Gupta. 2022. Code Smell Detection Using Ensemble Machine Learning Algorithms. Applied Sciences (Switzerland) 12, 20 (2022), 10321. DOI: 10.3390/app122010321

Dario Di Nucci, Fabio Palomba, Damian A. Tamburri, Alexander Serebrenik, and Andrea De Lucia. 2018. Detecting code smells using machine learning techniques: Are we there yet?, In IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER). 25th IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2018 - Proceedings 2018- March (2018), 612–621. DOI: 10.1109/SANER.2018.8330266

Katti Faceli, Ana Carolina Lorena, João Gama, Tiago Agostinho de Almeida, and André Carlos Ponce de Leon Ferreira de Carvalho. 2021. Inteligência artificial: uma abordagem de aprendizado de máquina. LTC, Brazil.

Vincenzo Ferme. 2013. JCodeOdor: A Software Quality Advisor Through Design Flaws Detection. Ph.D. Dissertation. Università degli Studi di Milano-Bicocca.

Francesca Arcelli Fontana, Vincenzo Ferme, Marco Zanoni, and Riccardo Roveda. 2015. Towards a prioritization of code debt: A code smell Intensity Index. In 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD) (7 ed.). IEEE, Bremen, Germany, 16–24. DOI: 10.1109/MTD.2015.7332620

Wenhua Hu, Lei Liu, Peixin Yang, Kuan Zou, Jiajun Li, Guancheng Lin, and Jianwen Xiang. 2023. Revisiting "code smell severity classification using machine learning techniques", In IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC), Shahriar H., Teranishi Y., Cuzzocrea A., Sharmin M., Towey D., Majumder AKM.J.A., Kashiwazaki H., Yang J.-J., Takemoto M., Sakib N., Banno R., and Ahamed S.I. (Eds.). Proceedings - International Computer Software and Applications Conference 2023-June (2023), 840–849. DOI: 10.1109/COMPSAC57700.2023.00113

Aleksandar Kovačević, Jelena Slivka, Dragan Vidaković, Katarina-Glorija Grujić, Nikola Luburić, Simona Prokić, and Goran Sladić. 2022. Automatic detection of Long Method and God Class code smells through neural source code embeddings. Expert Systems with Applications 204 (2022), 117607. DOI: 10.1016/j.eswa.2022.117607

Guilherme Lacerda, Fabio Petrillo, Marcelo Pimenta, and Yann Gaël Guéhéneuc. 2020. Code smells and refactoring: A tertiary systematic review of challenges and observations. Journal of Systems and Software 167 (2020), 110610. DOI: 10.1016/j.jss.2020.110610

Michele Lanza and Radu Marinescu. 2007. Object-oriented metrics in practice: using software metrics to characterize, evaluate, and improve the design of objectoriented systems. Springer Science & Business Media, Springer-Verlag Berlin Heidelberg 2006.

Mark Lorenz and Jeff Kidd. 1994. Object-oriented software metrics: a practical guide. Prentice-Hall, Inc., USA.

Lech Madeyski and Tomasz Lewowski. 2020. MLCQ: Industry-Relevant Code Smell Data Set. In Proceedings of the 24th International Conference on Evaluation and Assessment in Software Engineering (24 ed.) (Trondheim, Norway) (EASE ’20, 24). Association for Computing Machinery, New York, NY, USA, 342–347. DOI: 10.1145/3383219.3383264 03/04/2024.

Cristina Marinescu, Radu Marinescu, Petru Mihancea, Daniel Ratiu, and Richard Wettel. 2005. iPlasma: An Integrated Platform for Quality Assessment of Object-Oriented Design., In Proceedings of the 21st IEEE International Conference on Software Maintenance - Industrial and Tool volume, ICSM 2005, 25-30 September 2005, Budapest, Hungary. Proceedings of ICSM 1, 14, 77–80.

R. Marinescu. 2005. Measurement and quality in object-oriented design. In 21st IEEE International Conference on Software Maintenance (ICSM’05). IEEE, Budapest, Hungary, 701–704. DOI: 10.1109/ICSM.2005.63

T.J. McCabe. 1976. A Complexity Measure. IEEE Transactions on Software Engineering SE-2, 4 (1976), 308–320. DOI: 10.1109/TSE.1976.233837

Radu Marinescu Michele Lanza. 2006. Object-Oriented Metrics in Practice. Springer, Berlin, Heidelberg. XIV, 207 pages. DOI: 10.1007/3-540-39538-5

Naouel Moha, Yann-Gael Gueheneuc, Laurence Duchien, and Anne-Francoise Le Meur. 2010. DECOR: A Method for the Specification and Detection of Code and Design Smells. IEEE Transactions on Software Engineering 36, 1 (2010), 20–36. DOI: 10.1109/TSE.2009.50

Jatin Nanda and Jitender Kumar Chhabra. 2022. SSHM: SMOTE-stacked hybrid model for improving severity classification of code smell. International Journal of Information Technology (Singapore) 14, 5 (2022), 2701–2707. DOI: 10.1007/s41870-022-00943-8

Kwankamol Nongpong. 2012. Integrating "code smells" detection with refactoring tool support. Ph.D. Dissertation. University of Wisconsin at Milwaukee, USA. Advisor(s) Boyland, John Tang. AAI3523928.

Fabio Palomba, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Andrea De Lucia. 2014. Do They Really Smell Bad? A Study on Developers’ Perception of Bad Code Smells. In 2014 IEEE International Conference on Software Maintenance and Evolution. IEEE, Victoria, BC, Canada, 101–110. DOI: 10.1109/ICSME.2014.32

Fabio Palomba, Dario Di Nucci, Michele Tufano, Gabriele Bavota, Rocco Oliveto, Denys Poshyvanyk, and Andrea De Lucia. 2015. Landfill: An Open Dataset of Code Smells with Public Evaluation. In 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories (12 ed.). IEEE, Florence, Italy, 482–485. DOI: 10.1109/MSR.2015.69

Anubha Parashar, Apoorva Parashar, Weiping Ding, Mohammad Shabaz, and Imad Rida. 2023. Data preprocessing and feature selection techniques in gait recognition: A comparative study of machine learning and deep learning approaches. Pattern Recognition Letters 172 (2023), 65–73. DOI: 10.1016/j.patrec.2023.05.021

Archana Patnaik and Neelamadhab Padhy. 2022. Does Code Complexity Affect the Quality of Real-Time Projects? Detection of Code Smell on Software Projects Using Machine Learning Algorithms. In Proceedings of the International Conference on Data Science, Machine Learning and Artificial Intelligence (<confloc>, <city>Windhoek</city>, <country>Namibia</country>, </conf-loc>) (DSMLAI ’21’). Association for Computing Machinery, New York, NY, USA, 178–185. DOI: 10.1145/3484824.3484911

Fabiano Pecorelli, Fabio Palomba, Dario Di Nucci, and Andrea De Lucia. 2019. Comparing heuristic and machine learning approaches for metric-based code smell detection, In IEEE/ACM 27th International Conference on Program Comprehension (ICPC). IEEE International Conference on Program Comprehension 2019-May (2019), 93–104. DOI: 10.1109/ICPC.2019.00023

R. Polikar. 2006. Ensemble based systems in decision making. IEEE Circuits and Systems Magazine 6, 3 (2006), 21–45. DOI: 10.1109/MCAS.2006.1688199

Rajwant Singh Rao, Seema Dewangan, Alok Mishra, and Manjari Gupta. 2023. A study of dealing class imbalance problem with machine learning methods for code smell severity detection using PCA-based feature selection technique. Scientific Reports 13, 1 (2023). DOI: 10.1038/s41598-023-43380-8

Lior Rokach. 2010. Ensemble-based classifiers. Artificial Intelligence Review 33 (2010), 1–39.

Jelena Slivka, Nikola Luburić, Simona Prokić, Katarina-Glorija Grujić, Aleksandar Kovačević, Goran Sladić, and Dragan Vidaković. 2023. Towards a systematic approach to manual annotation of code smells. Science of Computer Programming 230 (2023), 102999. DOI: 10.1016/j.scico.2023.102999 20/03/2024.

Amjed Tahir, Jens Dietrich, Steve Counsell, Sherlock Licorish, and Aiko Yamashita. 2020. A large scale study on how developers discuss code smells and anti-pattern in Stack Exchange sites. Information and Software Technology 125 (2020), 106333. DOI: 10.1016/j.infsof.2020.106333 04/04/2024.

Ewan Tempero, Craig Anslow, Jens Dietrich, Ted Han, Jing Li, Markus Lumpe, Hayden Melton, and James Noble. 2010. The Qualitas Corpus: A Curated Collection of Java Code for Empirical Studies. In 2010 Asia Pacific Software Engineering Conference. IEEE, Sydney, NSW, Australia, 336–345. DOI: 10.1109/APSEC.2010.46

Nikolaos Tsantalis and Alexander Chatzigeorgiou. 2009. Identification of Move Method Refactoring Opportunities. IEEE Transactions on Software Engineering 35, 3 (2009), 347–367. DOI: 10.1109/TSE.2009.1

Nikolaos Tsantalis and Alexander Chatzigeorgiou. 2011. Ranking Refactoring Suggestions Based on Historical Volatility. In 2011 15th European Conference on Software Maintenance and Reengineering (15 ed.). IEEE, Oldenburg, Germany, 25–34. DOI: 10.1109/CSMR.2011.7

E. van Emden and L. Moonen. 2002. Java quality assurance by detecting code smells. In Ninth Working Conference on Reverse Engineering, 2002. Proceedings. IEEE, Richmond, VA, USA, 97–106. DOI: 10.1109/WCRE.2002.1173068

Santiago Vidal, Hernan Vazquez, J. Andres Diaz-Pace, Claudia Marcos, Alessandro Garcia, and Willian Oizumi. 2015. JSpIRIT: a flexible tool for the analysis of code smells. In 2015 34th International Conference of the Chilean Computer Science Society (SCCC). IEEE, Santiago, Chile, 1–6. DOI: 10.1109/SCCC.2015.7416572

Zhihong Wang, Hongru Ren, Renquan Lu, and Lirong Huang. 2022. Stacking Based LightGBM-CatBoost-RandomForest Algorithm and Its Application in Big Data Modeling. In 4th International Conference on Data-driven Optimization of Complex Systems (DOCS). IEEE, Chengdu, China, 1–6. DOI: 10.1109/DOCS55193.2022.9967714

Morteza Zakeri-Nasrabadi, Saeed Parsa, Ehsan Esmaili, and Fabio Palomba. 2023. A Systematic Literature Review on the Code Smells Datasets and Validation Mechanisms. ACM Comput. Surv. 55, 13s, Article 298 (jul 2023), 48 pages. DOI: 10.1145/3596908 02/03/2024.
Publicado
30/09/2024
SANTOS, Fábio do Rosario; DUARTE, Julio Cesar; CHOREN, Ricardo. Code smell severity classification at class and method level with a single manually labeled imbalanced dataset. In: SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE (SBES), 38. , 2024, Curitiba/PR. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2024 . p. 12-23. DOI: https://doi.org/10.5753/sbes.2024.2933.