Explainable Machine Learning Techniques for Criticality Prediction of Software Change Requests
Resumo
Text classification is an essential task in several real applications, commonly involving the use of Natural Language Processing (NLP) and Machine Learning (ML). Specifically, in the context of software testing, text classifiers have been applied to the analysis of Change Requests (CRs) documents, which are reports that report errors found in the software under test. Prioritizing critical CRs is crucial for more effectively maintaining the quality of developed products. This article explores the use of eXplainable AI (XAI) algorithms for the inspection of CR criticality classifiers, facilitating the interpretation and justification of decisions made by the ML model. Once a CR is classified as critical, explanations are generated to identify important characteristics that help understand why the error reported by the CR was considered critical. In a case study, we implemented the LIME technique to provide interpretable explanations of the predictions made by an XGBoost model, applied to a database extracted from a real industrial context of mobile device development. The results show that XAI can increase confidence in ML model decisions by highlighting the most significant terms in critical and non-critical CRs.
Referências
Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321–357.
Fisher, A., Rudin, C., and Dominici, F. (2019). All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. Journal of Machine Learning Research, 20(177):1–81.
Gomes, L. A. F., da Silva Torres, R., and Côrtes, M. L. (2019). Bug report severity level prediction in open source software: A survey and research opportunities. Information and Software Technology, 115:58–78.
He, H., Bai, Y., Garcia, E. A., and Li, S. (2008). Adasyn: Adaptive synthetic sampling approach for imbalanced learning. In 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), pages 1322–1328. Ieee.
Lamkanfi, A., Demeyer, S., Giger, E., and Goethals, B. (2010). Predicting the severity of a reported bug. In 2010 7th IEEE working conference on mining software repositories (MSR 2010), pages 1–10. IEEE.
Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Mohammed, Roweida e Rawashdeh, J. e. A. M. (2020). Aprendizado de máquina com técnicas de sobreamostragem e subamostragem: estudo geral e resultados experimentais. In 2020 11ª Conferência Internacional sobre Sistemas de Informação e Comunicação (ICICS), pages 243–248.
Molnar, C. (2020). Interpretable machine learning. [link].
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). ”why should i trust you?”: Explaining the predictions of any classifier.
XGBoost Developers (2022). Xgboost parameters. [link], Acessado em 2024-02-06.
Zahoor, K., Bawany, N., and Qamar, T. (2024). Evaluating text classification with explainable artificial intelligence. IAES International Journal of Artificial Intelligence (IJ-AI), 13:278–286.
Zhang, J., Wang, X., Hao, D., Xie, B., Zhang, L., and Mei, H. (2015). A survey on bug-report analysis. Sci. China Inf. Sci., 58(2):1–24.
