Environment for the Evaluation of Functional Adequacy in AI Systems

• Jesús Oviedo Lama AQCLab Software Quality / Universidad de Castilla-La Mancha
• Moisés Rodríguez Monje AQCLab Software Quality / Universidad de Castilla-La Mancha
• Mario Piattini Velthuis Universidad de Castilla-La Mancha

Abstract

Artificial Intelligence is currently a fundamental part of the digital transformation of organizations and the appearance of applications with AI algorithms means that their impact on everyday activities is increasing every day. In this situation, there is a growing need for these AI systems, as software systems, to have the necessary quality parameters to guarantee their use. This article presents a methodological and technological environment for the measurement and evaluation of Functional Suitability in AI Systems. The environment proposes a set of metrics and quality properties aligned with the new ISO/IEC 25059 standard. It also has an assessment methodology aligned with ISO/IEC 25040. On the other hand, a set of automatic tools to facilitate the evaluation of the quality of Functional in IA Systems.

References

Calero, C., Moraga, M. Á., and Piattini, M. (2021). Software Sustainability. Springer.

Horneman, A., Mellinger, A., and Ozkaya, I. (2019). Ai engineering: 11 foundational practices-recommendations for decision makers from experts in software engineering, cybersecurity, and applied artificial intelligence white paper dm19-0624, 06.06.

ISO/IEC (2023a). Iso/iec 25040:2023. In Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Evaluation process.

ISO/IEC (2023b). Iso/iec 25059:2023. In Software engineering - Systems and software Quality Requirements and Evaluations (Square) - Quality model for AI systems.

Lavazza, L. and Morasca, S. (2021). Understanding and modeling ai-intensive system development. In 2021 IEEE/ACM 1st Workshop on AI Engineering-Software Engineering for AI (WAIN), pages 55–61. IEEE.

Navas, R. (2016). Modelo de calidad para servicios cloud.

Oviedo, J., Márquez, R., Rodríguez, M., and Piattini, M. (2023). Calidad en los sistemas ia: adaptación de modelos de procesos y productos. Actas de las XXVII Jornadas de Ingeniería del Software y Bases de Datos.

Rahman, A. (2019). Quality consideration for e-learning system based on iso/iec 25000 quality standard.

Rodríguez, M., Oviedo, J. R., and Piattini, M. (2016). Evaluation of software product functional suitability: A case study. Software Quality Professional, 18(3).

Romero, J., Medina-Bulo, I., and Chicano, F. (2023). Optimising the Software Development Process with Artificial Intelligence. Springer.

Van Oort, B., Cruz, L., Aniche, M., and Van Deursen, A. (2021). The prevalence of code smells in machine learning projects. In 2021 IEEE/ACM 1st Workshop on AI Engineering-Software Engineering for AI (WAIN), pages 1–8. IEEE.

Verdugo, J., Oviedo, J., Rodríguez, M., and Piattini, M. (2024). Connecting research and practice for software product quality evaluation and certification: a software laboratory’s 25-year journey. IEEE Software.