Requirements Engineering for ML-Enabled Systems: Status Quo and Problems

• Antonio Pedro Santos Alves PUC-Rio
• Marcos Kalinowski PUC-Rio
• Daniel Méndez BTH

Resumo

Machine Learning (ML) systems are increasingly common in companies seeking to improve products and processes. While literature suggests that Requirements Engineering (RE) can address challenges in ML-enabled systems, most empirical studies are isolated and lack generalizability. The goal of this dissertation is to enhance the empirical evidence on the intersection of RE and ML-enabled systems. For this purpose, we conducted an international survey with 188 respondents from 25 countries, using statistical and qualitative analyses to explore RE practices and challenges in ML projects. Our key findings include: (i) project leaders and data scientists primarily handle RE activities, (ii) interactive Notebooks are the dominant documentation format, (iii) data quality, model reliability, and explainability are the main non-functional requirements, (iv) challenges when developing such systems include managing customer expectations and aligning requirements with data, and (v) the main problems practitioners face are related to lack of business domain understanding, unclear goals and requirements, and low customer engagement. These results give us a wider picture of the adopted practices and the challenges in industrial scenarios. We put forward the need to adapt further and disseminate RE-related practices for engineering high-quality ML-enabled systems.

Palavras-chave: Requirements Engineering, Machine Learning, Survey