Deployment of a Machine Learning System for Predicting Lawsuits Against Power Companies: Lessons Learned from an Agile Testing Experience for Improving Software Quality

• Luis Rivero UFMA https://orcid.org/0000-0001-6008-6537
• João Diniz UFMA
• Giovanni Silva UFMA
• Gabriel Borralho UFMA
• Geraldo Braz UFMA https://orcid.org/0000-0003-3731-6431
• Anselmo Paiva UFMA https://orcid.org/0000-0003-4921-0626
• Erika Alves Equatorial Energia S/A
• Milton Oliveira Equatorial Energia S/A

Resumo

The advances in Machine Learning (ML) require software organizations to evolve their development processes in order to improve the quality of ML systems. Within the software development process, the testing stage of an ML system is more critical, considering that it is necessary to add data validation, trained model quality evaluation, and model validation to traditional unit, integration tests and system tests. In this paper, we focus on reporting the lessons learned of using model testing and exploratory testing within the context of the agile development process of an ML system that predicts lawsuits proneness in energy supply companies. Through the development of the project, the SCRUM agile methodology was applied and activities related to the development of the ML model and the development of the end-user application were defined. After the testing process of the ML model, we managed to achieve 93.89 accuracy; 95.58 specificity; 88.84 sensitivity; and 87.09 precision. Furthermore, we focused on the quality of use of the application embedding the ML model, by carrying out exploratory testing. As a result, through several iterations, different types of defects were identified and corrected. Our lessons learned support software engineers willing to develop ML systems that consider both the ML model and the end-user application.

Palavras-chave: Software Processes, Methods, and Tools, Verification, Validation, and Testing