Machine Learning for the Identification and Classification of Technical Debt Types on StackOverflow Discussions
Resumo
In today’s fast-paced software industry, understanding and managing Technical Debt (TD) is crucial for software development. TD can compromise the long-term quality of software systems. The occurrence of TD is commonly reported and discussed by practitioners on Question and Answers (Q&A) platforms, such as Stack Overflow (SO). Data from Q&A platforms has been leveraged by the TD research community, most prominently regarding knowledge extraction. However, manual analyses of such data not only require considerable effort but also suffer from biases. Hence, this paper aims to propose an automated approach for identifying and classifying types of TD in SO discussions using machine learning (ML) and natural language processing. We divided our methodology into four main steps: i) data preprocessing, ii) application of natural language processing, iii) application of ML algorithms, and iv) computing the evaluation metrics for the proposed models. Our results indicate that ML algorithms have the potential to be successfully applied to automatically identify and classify TD types on SO discussions.We achieved a recall of 85% for test debt and a precision of 78% for design debt. Furthermore, the results of automated TD identification on SO benefit the software development community by enhancing solution quality, raising awareness of best practices, and facilitating collaboration among developers. This leads to more efficient development and the promotion of consistent standards. We make our entire dataset and pre-trained models available to encourage future research directions.
Palavras-chave:
Technical Debt, Stack Overflow, Machine Learning
Referências
Arshad Ahmad, Chong Feng, Shi Ge, and Abdallah Yousif. 2018. A survey on mining stack overflow: question and answering (Q&A) community. Data Technologies and Applications 52, 2 (2018), 190–247.
Danyllo Albuquerque, Everton Guimarães, Graziela Tonin, Pilar Rodríguez, Mirko Barbosa Perkusich, Hyggo O. Almeida, Angelo Perkusich, and Ferdinandy Chagas. 2023. Managing Technical Debt Using Intelligent Techniques - A Systematic Mapping Study. IEEE Trans. Software Eng. 49, 4 (2023), 2202–2220. https://doi.org/10.1109/TSE.2022.3214764
Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2012. Discovering value from community activity on focused question answering sites. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '12. ACM Press.
Alan Bandeira, Carlos Alberto Medeiros, Matheus Paixao, and Paulo Henrique Maia. 2019. We Need to Talk about Microservices: an Analysis from the Discussions on StackOverflow. International Conference on Mining Software Repositories 5 (2019), 255–259.
Xiaoyun Cheng, Naming Liu, Lin Guo, Zhou Xu, and Tao Zhang. 2020. Blocking Bug Prediction Based on XGBoost with Enhanced Features. In 44th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2020, Madrid, Spain, July 13-17, 2020. IEEE, 902–911. https://doi.org/10.1109/COMPSAC48688.2020.0-152
Diego Costa, Mariela Inés Cortés, and Eliakim Gama. 2021. On the relation between technical debt indicators and quality criteria in Stack Overflow discussions. In SBES ’21: 35th Brazilian Symposium on Software Engineering, Joinville, Santa Catarina, Brazil, 27 September 2021 - 1 October 2021. ACM, 432–441.
Michael A. Covington, Ken Barker, and Stan Szpakowicz. 1994. Natural Language Processing for Prolog Programmers.
Ward Cunningham. 1992. The WyCash Portfolio Management System. In Addendum to the Proceedings on Object-oriented Programming Systems, Languages, and Applications (Addendum). 29–30.
Everton da S. Maldonado, Emad Shihab, and Nikolaos Tsantalis. 2017. Using Natural Language Processing to Automatically Detect Self-Admitted Technical Debt. IEEE Trans. Software Eng. 43, 11 (2017), 1044–1062.
Eliakim Gama. 2021. Investigando como os profissionais do Stack Overflow discutem sobre dívida técnica e a sua identificação em projetos de software. In Dissertação (Mestrado Acadêmico em Ciências da Computação) - Programa De Pós-Graduação Em Ciência Da Computação - Universidade Estadual do Ceará.
Eliakim Gama, Mariela I. Cortes, Matheus Paixao, and Adson Damasceno. 2023. Replication package for the paper: “Machine Learning for the Identification and Classification of Technical Debt Types on StackOverflow Discussions”. https://zenodo.org/record/8331281
Eliakim Gama, Sávio Freire, Manoel Mendonça, Rodrigo O. Spínola, Matheus Paixao, and Mariela I. Cortés. 2020. Using StackOverflowtoAssess Technical Debt Identification on Software Projects. In Proceedings of the 34th Brazilian Symposium on Software Engineering (SBES ’20). Association for Computing Machinery, New York, NY, USA, 730–739.
Eliakim Gama, Matheus Paixao, Emmanuel Sávio Silva Freire, and Mariela Inés Cortés. 2019. Technical Debt’s State of Practice on Stack Overflow: A Preliminary Study. In Proceedings of the XVIII Brazilian Symposium on Software Quality (SBQS’19). Association for Computing Machinery, New York, NY, USA, 228–233.
Jiankun Ge, Linfeng Zhao, Zihui Yu, Huanhuan Liu, Lei Zhang, Xuewen Gong, and Huaiwei Sun. 2022. Prediction of Greenhouse Tomato Crop Evapotranspiration Using XGBoost Machine Learning Model. Plants 11, 15 (2022).
M. Firdaus Harun and Horst Lichter. 2015. Towards a Technical Debt Management Framework based on Cost-Benefit Analysis. In Proceedings of The Tenth International Conference on Software Engineering Advances. 70–73.
Khalil M.El Hindi, Reem R.Aljulaidan, and HussienAlSalman. 2020. Lazy finetuning algorithms for naïve Bayesian text classification. In Journal Elsevier.
Ibrahim Karabayir, Samuel M. Goldman, Suguna Pappu, and Oguz Akbilgic. 2020. Gradient boosting for Parkinson’s disease diagnosis from voice recordings. In MC Medical Informatics and Decision Making, vol. 20.
Anshu Khurana and Om Prakash Verma. 2020. Novel approach with natureinspired and ensemble techniques for optimal text classification. In Multimed Tools Appl 79.
Nicholas Kozanidis, Roberto Verdecchia, and Emitzá Guzmán. 2022. Asking about Technical Debt Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow. In International Symposium on Empirical Software Engineering and Measurement (ESEM) 2022.
Max Kuhn and Kjell Johnson. 2013. Applied predictive modeling. http://www.amazon.com/Applied-Predictive-Modeling-Max-Kuhn/dp/1461468485/.
Zengyang Li, Paris Avgeriou, and Peng Liang. 2015. A Systematic Mapping Study on Technical Debt and Its Management. J. Syst. Softw. 101, C (March 2015).
R.C. Lupton and J.M. Allwood. 2017. Hybrid Sankey diagrams: Visual analysis of multidimensional data for understanding resource use. Resources, Conservation and Recycling 124 (2017), 141–151. https://doi.org/10.1016/j.resconrec.2017.05.002
Amit Kumar Nandanwar and Jaytrilok Choudhary. 2021. Semantic Features with Contextual Knowledge-Based Web Page Categorization Using the GloVe Model and Stacked BiLSTM. Symmetry 13, 10 (2021). https://www.mdpi.com/2073-8994/13/10/1772
Dragutin Petkovic, Marc Sosnick-Pérez, Kazunori Okada, Rainer Todtenhoefer, Shihong Huang, Nidhi Miglani, and Arthur Vigil. 2016. Using the random forest classifier to assess and predict student learning of Software Engineering Teamwork. In 2016 IEEE Frontiers in Education Conference, FIE 2015, Eire, PA, USA, October 12-15, 2016. IEEE Computer Society, 1–7.
Luca Ponzanelli, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Michele Lanza. 2014. Mining StackOverflow to turn the IDE into a self-confident programming prompter. In Proceedings of the 11th Working Conference on Mining Software Repositories. ACM, 102–111.
S. Poornima and M. Pushpalatha. 2018. A survey of predictive analytics using big data with data mining.
Anand Rajaraman and Jeffrey David Ullman. 2011. Data Mining. In Cambridge University Press.
Narayan Ramasubbu and Chris Kemerer. 2018. Integrating Technical Debt Management and Software Quality Management Processes: A Framework and Field Tests. In Proceedings of the 40th International Conference on Software Engineering.
Leevi Rantala. 2020. Towards better technical debt detection with NLP and machine learning methods. In ICSE ’20: 42nd International Conference on Software Engineering, Companion Volume, Seoul, South Korea, 27 June - 19 July, 2020, Gregg Rothermel and Doo-Hwan Bae (Eds.). ACM, 242–245.
Xiaoxue Ren, Zhenchang Xing, Xin Xia, David Lo, XinyuWang, and John Grundy and. 2019. Neural Network-based Detection of Self-Admitted Technical Debt: From Performance to Explainability. In ACM Transactions on Software Engineering and Methodology.
Nicolli Rios, Manoel Gomes de Mendonça Neto, and Rodrigo Oliveira Spínola. 2018. A tertiary study on technical debt: Types, management strategies, research trends, and base information for practitioners. Information and Software Technology 102 (2018), 117 – 145.
Christoffer Rosen and Emad Shihab. 2016. What are mobile developers asking about? A large scale study using stack overflow. Empirical Software Engineering 21, 3 (jun 2016), 1192–1223.
Riccardo Rubei, Claudio Di Sipio, Phuong Thanh Nguyen, Juri Di Rocco, and Davide Di Ruscio. 2020. PostFinder: Mining Stack Overflow posts to support software developers. Inf. Softw. Technol. 127 (2020), 106367.
Amandeep Kaur Sandhu and Ranbir Singh Batth. 2021. Software reuse analytics using integrated random forest and gradient boosting machine learning algorithm. Softw. Pract. Exp. 51, 4 (2021), 735–747. https://doi.org/10.1002/spe.2921
StackOverflow. [n.d.]. Stack Overflow Annual Developer Survey. https://insights.stackoverflow.com/survey/ Acessado: 2019-07-23.
Christoph Treude and Martin P Robillard. 2016. Augmenting api documentation with insights from stack overflow. In 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE). IEEE, 392–403.
Fei Xing, Ping Guo, and Michael R. Lyu. 2005. A Novel Method for Early Software Quality Prediction Based on Support Vector Machine. In 16th International Symposium on Software Reliability Engineering (ISSRE 2005), 8-11 November 2005, Chicago, IL, USA. IEEE Computer Society, 213–222.
Cheng-Zen Yang, Chun-Chi Hou, Wei-Chen Kao, and Ing-Xiang Chen. 2012. An Empirical Study on Improving Severity Prediction of Defect Reports Using Feature Selection. In 19th Asia-Pacific Software Engineering Conference, APSEC 2012, Hong Kong, China, December 4-7, 2012, Karl R. P. H. Leung and Pornsiri Muenchaisri (Eds.). IEEE, 240–249.
Danyllo Albuquerque, Everton Guimarães, Graziela Tonin, Pilar Rodríguez, Mirko Barbosa Perkusich, Hyggo O. Almeida, Angelo Perkusich, and Ferdinandy Chagas. 2023. Managing Technical Debt Using Intelligent Techniques - A Systematic Mapping Study. IEEE Trans. Software Eng. 49, 4 (2023), 2202–2220. https://doi.org/10.1109/TSE.2022.3214764
Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2012. Discovering value from community activity on focused question answering sites. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '12. ACM Press.
Alan Bandeira, Carlos Alberto Medeiros, Matheus Paixao, and Paulo Henrique Maia. 2019. We Need to Talk about Microservices: an Analysis from the Discussions on StackOverflow. International Conference on Mining Software Repositories 5 (2019), 255–259.
Xiaoyun Cheng, Naming Liu, Lin Guo, Zhou Xu, and Tao Zhang. 2020. Blocking Bug Prediction Based on XGBoost with Enhanced Features. In 44th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2020, Madrid, Spain, July 13-17, 2020. IEEE, 902–911. https://doi.org/10.1109/COMPSAC48688.2020.0-152
Diego Costa, Mariela Inés Cortés, and Eliakim Gama. 2021. On the relation between technical debt indicators and quality criteria in Stack Overflow discussions. In SBES ’21: 35th Brazilian Symposium on Software Engineering, Joinville, Santa Catarina, Brazil, 27 September 2021 - 1 October 2021. ACM, 432–441.
Michael A. Covington, Ken Barker, and Stan Szpakowicz. 1994. Natural Language Processing for Prolog Programmers.
Ward Cunningham. 1992. The WyCash Portfolio Management System. In Addendum to the Proceedings on Object-oriented Programming Systems, Languages, and Applications (Addendum). 29–30.
Everton da S. Maldonado, Emad Shihab, and Nikolaos Tsantalis. 2017. Using Natural Language Processing to Automatically Detect Self-Admitted Technical Debt. IEEE Trans. Software Eng. 43, 11 (2017), 1044–1062.
Eliakim Gama. 2021. Investigando como os profissionais do Stack Overflow discutem sobre dívida técnica e a sua identificação em projetos de software. In Dissertação (Mestrado Acadêmico em Ciências da Computação) - Programa De Pós-Graduação Em Ciência Da Computação - Universidade Estadual do Ceará.
Eliakim Gama, Mariela I. Cortes, Matheus Paixao, and Adson Damasceno. 2023. Replication package for the paper: “Machine Learning for the Identification and Classification of Technical Debt Types on StackOverflow Discussions”. https://zenodo.org/record/8331281
Eliakim Gama, Sávio Freire, Manoel Mendonça, Rodrigo O. Spínola, Matheus Paixao, and Mariela I. Cortés. 2020. Using StackOverflowtoAssess Technical Debt Identification on Software Projects. In Proceedings of the 34th Brazilian Symposium on Software Engineering (SBES ’20). Association for Computing Machinery, New York, NY, USA, 730–739.
Eliakim Gama, Matheus Paixao, Emmanuel Sávio Silva Freire, and Mariela Inés Cortés. 2019. Technical Debt’s State of Practice on Stack Overflow: A Preliminary Study. In Proceedings of the XVIII Brazilian Symposium on Software Quality (SBQS’19). Association for Computing Machinery, New York, NY, USA, 228–233.
Jiankun Ge, Linfeng Zhao, Zihui Yu, Huanhuan Liu, Lei Zhang, Xuewen Gong, and Huaiwei Sun. 2022. Prediction of Greenhouse Tomato Crop Evapotranspiration Using XGBoost Machine Learning Model. Plants 11, 15 (2022).
M. Firdaus Harun and Horst Lichter. 2015. Towards a Technical Debt Management Framework based on Cost-Benefit Analysis. In Proceedings of The Tenth International Conference on Software Engineering Advances. 70–73.
Khalil M.El Hindi, Reem R.Aljulaidan, and HussienAlSalman. 2020. Lazy finetuning algorithms for naïve Bayesian text classification. In Journal Elsevier.
Ibrahim Karabayir, Samuel M. Goldman, Suguna Pappu, and Oguz Akbilgic. 2020. Gradient boosting for Parkinson’s disease diagnosis from voice recordings. In MC Medical Informatics and Decision Making, vol. 20.
Anshu Khurana and Om Prakash Verma. 2020. Novel approach with natureinspired and ensemble techniques for optimal text classification. In Multimed Tools Appl 79.
Nicholas Kozanidis, Roberto Verdecchia, and Emitzá Guzmán. 2022. Asking about Technical Debt Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow. In International Symposium on Empirical Software Engineering and Measurement (ESEM) 2022.
Max Kuhn and Kjell Johnson. 2013. Applied predictive modeling. http://www.amazon.com/Applied-Predictive-Modeling-Max-Kuhn/dp/1461468485/.
Zengyang Li, Paris Avgeriou, and Peng Liang. 2015. A Systematic Mapping Study on Technical Debt and Its Management. J. Syst. Softw. 101, C (March 2015).
R.C. Lupton and J.M. Allwood. 2017. Hybrid Sankey diagrams: Visual analysis of multidimensional data for understanding resource use. Resources, Conservation and Recycling 124 (2017), 141–151. https://doi.org/10.1016/j.resconrec.2017.05.002
Amit Kumar Nandanwar and Jaytrilok Choudhary. 2021. Semantic Features with Contextual Knowledge-Based Web Page Categorization Using the GloVe Model and Stacked BiLSTM. Symmetry 13, 10 (2021). https://www.mdpi.com/2073-8994/13/10/1772
Dragutin Petkovic, Marc Sosnick-Pérez, Kazunori Okada, Rainer Todtenhoefer, Shihong Huang, Nidhi Miglani, and Arthur Vigil. 2016. Using the random forest classifier to assess and predict student learning of Software Engineering Teamwork. In 2016 IEEE Frontiers in Education Conference, FIE 2015, Eire, PA, USA, October 12-15, 2016. IEEE Computer Society, 1–7.
Luca Ponzanelli, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Michele Lanza. 2014. Mining StackOverflow to turn the IDE into a self-confident programming prompter. In Proceedings of the 11th Working Conference on Mining Software Repositories. ACM, 102–111.
S. Poornima and M. Pushpalatha. 2018. A survey of predictive analytics using big data with data mining.
Anand Rajaraman and Jeffrey David Ullman. 2011. Data Mining. In Cambridge University Press.
Narayan Ramasubbu and Chris Kemerer. 2018. Integrating Technical Debt Management and Software Quality Management Processes: A Framework and Field Tests. In Proceedings of the 40th International Conference on Software Engineering.
Leevi Rantala. 2020. Towards better technical debt detection with NLP and machine learning methods. In ICSE ’20: 42nd International Conference on Software Engineering, Companion Volume, Seoul, South Korea, 27 June - 19 July, 2020, Gregg Rothermel and Doo-Hwan Bae (Eds.). ACM, 242–245.
Xiaoxue Ren, Zhenchang Xing, Xin Xia, David Lo, XinyuWang, and John Grundy and. 2019. Neural Network-based Detection of Self-Admitted Technical Debt: From Performance to Explainability. In ACM Transactions on Software Engineering and Methodology.
Nicolli Rios, Manoel Gomes de Mendonça Neto, and Rodrigo Oliveira Spínola. 2018. A tertiary study on technical debt: Types, management strategies, research trends, and base information for practitioners. Information and Software Technology 102 (2018), 117 – 145.
Christoffer Rosen and Emad Shihab. 2016. What are mobile developers asking about? A large scale study using stack overflow. Empirical Software Engineering 21, 3 (jun 2016), 1192–1223.
Riccardo Rubei, Claudio Di Sipio, Phuong Thanh Nguyen, Juri Di Rocco, and Davide Di Ruscio. 2020. PostFinder: Mining Stack Overflow posts to support software developers. Inf. Softw. Technol. 127 (2020), 106367.
Amandeep Kaur Sandhu and Ranbir Singh Batth. 2021. Software reuse analytics using integrated random forest and gradient boosting machine learning algorithm. Softw. Pract. Exp. 51, 4 (2021), 735–747. https://doi.org/10.1002/spe.2921
StackOverflow. [n.d.]. Stack Overflow Annual Developer Survey. https://insights.stackoverflow.com/survey/ Acessado: 2019-07-23.
Christoph Treude and Martin P Robillard. 2016. Augmenting api documentation with insights from stack overflow. In 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE). IEEE, 392–403.
Fei Xing, Ping Guo, and Michael R. Lyu. 2005. A Novel Method for Early Software Quality Prediction Based on Support Vector Machine. In 16th International Symposium on Software Reliability Engineering (ISSRE 2005), 8-11 November 2005, Chicago, IL, USA. IEEE Computer Society, 213–222.
Cheng-Zen Yang, Chun-Chi Hou, Wei-Chen Kao, and Ing-Xiang Chen. 2012. An Empirical Study on Improving Severity Prediction of Defect Reports Using Feature Selection. In 19th Asia-Pacific Software Engineering Conference, APSEC 2012, Hong Kong, China, December 4-7, 2012, Karl R. P. H. Leung and Pornsiri Muenchaisri (Eds.). IEEE, 240–249.
Publicado
26/09/2023
Como Citar
GAMA, Eliakim; CORTÉS, Mariela I.; PAIXAO, Matheus; DAMASCENO, Adson.
Machine Learning for the Identification and Classification of Technical Debt Types on StackOverflow Discussions. In: WORKSHOP BRASILEIRO DE ENGENHARIA DE SOFTWARE INTELIGENTE (ISE), 3. , 2023, Campo Grande/MS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 25-30.
DOI: https://doi.org/10.5753/ise.2023.235840.
