Identifying Concerns When Specifying Machine Learning-Enabled Systems: A Perspective-Based Approach

• Hugo Villamizar PUC-Rio
• Marcos Kalinowski PUC-Rio

Resumo

Engineering successful machine learning (ML)-enabled systems poses various challenges from both a theoretical and a practical side. Among those challenges are how to effectively address unrealistic expectations of ML capabilities from customers, managers and even other team members, and how to connect business value to engineering and data science activities composed by interdisciplinary teams. In this thesis, we studied the state of the practice and literature of requirements engineering (RE) for ML to propose PerSpecML, a perspective-based approach for specifying ML-enabled systems that helps practitioners identify which attributes, including ML and non-ML components, are important to contribute to the overall system’s quality. The approach involves analyzing 60 concerns related to 28 tasks that practitioners typically face in ML projects, grouping them into five perspectives: system objectives, user experience, infrastructure, model, and data. The conception of  PerSpecML involved a series of validations conducted in different contexts: (i) in academia, (ii) with industry representatives, and (iii) in two real industrial case studies. As a result of the diverse validations and continuous improvements, PerSpecML showed a positive impact to the specification of ML-enabled systems, particularly helping to specify key quality components that would have been otherwise missed without using PerSpecML.

Palavras-chave: Requirements Specification, Machine Learning, Software Quality