Testing Non-Functional Requirements: Lacking of Technologies or Researching Opportunities?
Resumo
Software testing aims to reveal failures due to the lack of conformity (defects) among functional and non-functional requirements and the implemented system. Thus, defects can be identified and fixed, improving software quality. However, despite several works emphasizing the importance of non-functional requirements (NFRs), there is an insufficient amount of software testing approaches dealing with them. The lack of NFR evaluation may be the cause of low-quality software that does not meet users need, influencing software project success. Goal: To organize a body of knowledge regarding NFRs and software testing approaches available in the technical literature and reveal the gaps between testable NFRs and software testing approaches. Method: To perform structured literature reviews to identify NFRs and software testing approaches dealing with testable NFRs. To combine both results, reveal research opportunities and organize a body of knowledge regarding NFRs and software testing approaches. Results: From 224 identified NFRs, 87 were described, and 47 software testing approaches observed. Only eight approaches are empirically evaluated. No testing approaches were identified for 11 testable NFRs. Furthermore, regarding the testing process, we did not observe any testing approach covering the test planning phase. Conclusion: Despite their importance, many testable NFRs seem not be tested due to the lack of appropriate software testing approaches yet. Also, the existing testing approaches do not cover all testing processes activities and, in general, lack empirical evidence about their feasibility and performance, making their use in software projects risky.
Palavras-chave:
Non-Functional, Requirements, Researching
Referências
Ameller, D., Galster, M., Avgeriou, P. and Franch, X. (2015) “A survey on quality attributes in service-based systems, Software Quality Journal”, Kluwer Academic Publishers.
Ameller, D., Ayala, C., Cabot, J. and Franch, X. (2012) “How do software architects consider non-functional requirements: an exploratory study”, 20th IEEE International Requirements Engineering Conference, 41–50.
Bajpai, V. and Gorthi, R. (2012) “On non-functional requirements: a survey”, Conference on Electrical, Electronics and Computer Science: Innovation for Humanity, SCEECS.
Borg, A., Yong, A., Carlshamre, P. and Sandahl, K. (2003) “The bad conscience of requirements engineering: an investigation in real-world treatment of non-functional requirements”, 3rd Conference on Software Engineering Research and Practice in Sweden (SERPS).
Chung, L. and do Prado Leite, J. C. S. (2009) “On non-Functional Requirements in Software Engineering”, In conceptual modeling: foundations and applications, 363379. DOI=10.1007/978-3-642-02463-4_19.
Eckhardt, J., Vogelsang, A. and Fernández, D. M. (2016) “Are ‘non-functional’ requirements really non-functional?”, 38th International Conference on Software Engineering (Austin, Texas).
Glinz, M. (2007) “On non-functional requirements”, 15th IEEE International Requirements Engineering Conference, 21-26.
ISO/IEC/IEEE 29119-1. (2013) “Software and systems engineering software testing”.
Joorabchi, M. E., Mesbah, A. and Kruchten, P. (2013) “Real challenges in mobile app development”, ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 15-24. DOI=http://dx.doi.org/10.1109/ESEM.2013.9.
Mairiza, D., Zowghi, D. and Nurmuliani, N. (2010) “An investigation into the notion of non-functional requirements”, Conference of 25th Annual ACM Symposium on Applied Computing. (March 2010). 311-317.
Montagud, S., Abrahão, S. and Insfran, E. (2012) “A systematic review of quality attributes and measures for software product lines”, Software Quality Journal, Vol. 20 (3-4), pp. 425-486.
Poort, E.R., Martens, N., Van De Weerd, I. and Van Vliet, H. (2012) “How architects see non-functional requirements: Beware of modifiability”, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 37-55.
Rashid, M., Ardito, L. and Torchiano, M. (2015) “Energy consumption analysis of algorithms implementations”, ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1-4. DOI= http://dx.doi.org/10.1109/ESEM.2015.7321198.
Strauss, A. and Corbin, J. (1998) “Basics of qualitative research: techniques and procedures for developing grounded theory”, SAGE Publications, 2nd Ed. London.
Svensson, R. B., Gorschek, T. and Regnell. B. (2009) “Quality requirements in practice: An interview study in requirements engineering for embedded systems”, Foundation for Software Quality, volume 5512 of Lecture Notes in Computer Science. Springer.
van Heesch, U., Avgeriou, P. (2011) “Mature architecting a survey about the reasoning process of professional architects”, Software Architecture (WICSA), pp. 260–269.
Yang, Z., Li, Z. C., Jin, Z. and Chen, Y. (2014) “A systematic literature review of requirements modeling and analysis for self-adaptive systems”, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer Verlag, Vol. 8396 LNCS, 55-71.
Ameller, D., Ayala, C., Cabot, J. and Franch, X. (2012) “How do software architects consider non-functional requirements: an exploratory study”, 20th IEEE International Requirements Engineering Conference, 41–50.
Bajpai, V. and Gorthi, R. (2012) “On non-functional requirements: a survey”, Conference on Electrical, Electronics and Computer Science: Innovation for Humanity, SCEECS.
Borg, A., Yong, A., Carlshamre, P. and Sandahl, K. (2003) “The bad conscience of requirements engineering: an investigation in real-world treatment of non-functional requirements”, 3rd Conference on Software Engineering Research and Practice in Sweden (SERPS).
Chung, L. and do Prado Leite, J. C. S. (2009) “On non-Functional Requirements in Software Engineering”, In conceptual modeling: foundations and applications, 363379. DOI=10.1007/978-3-642-02463-4_19.
Eckhardt, J., Vogelsang, A. and Fernández, D. M. (2016) “Are ‘non-functional’ requirements really non-functional?”, 38th International Conference on Software Engineering (Austin, Texas).
Glinz, M. (2007) “On non-functional requirements”, 15th IEEE International Requirements Engineering Conference, 21-26.
ISO/IEC/IEEE 29119-1. (2013) “Software and systems engineering software testing”.
Joorabchi, M. E., Mesbah, A. and Kruchten, P. (2013) “Real challenges in mobile app development”, ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 15-24. DOI=http://dx.doi.org/10.1109/ESEM.2013.9.
Mairiza, D., Zowghi, D. and Nurmuliani, N. (2010) “An investigation into the notion of non-functional requirements”, Conference of 25th Annual ACM Symposium on Applied Computing. (March 2010). 311-317.
Montagud, S., Abrahão, S. and Insfran, E. (2012) “A systematic review of quality attributes and measures for software product lines”, Software Quality Journal, Vol. 20 (3-4), pp. 425-486.
Poort, E.R., Martens, N., Van De Weerd, I. and Van Vliet, H. (2012) “How architects see non-functional requirements: Beware of modifiability”, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 37-55.
Rashid, M., Ardito, L. and Torchiano, M. (2015) “Energy consumption analysis of algorithms implementations”, ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1-4. DOI= http://dx.doi.org/10.1109/ESEM.2015.7321198.
Strauss, A. and Corbin, J. (1998) “Basics of qualitative research: techniques and procedures for developing grounded theory”, SAGE Publications, 2nd Ed. London.
Svensson, R. B., Gorschek, T. and Regnell. B. (2009) “Quality requirements in practice: An interview study in requirements engineering for embedded systems”, Foundation for Software Quality, volume 5512 of Lecture Notes in Computer Science. Springer.
van Heesch, U., Avgeriou, P. (2011) “Mature architecting a survey about the reasoning process of professional architects”, Software Architecture (WICSA), pp. 260–269.
Yang, Z., Li, Z. C., Jin, Z. and Chen, Y. (2014) “A systematic literature review of requirements modeling and analysis for self-adaptive systems”, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer Verlag, Vol. 8396 LNCS, 55-71.
Publicado
24/10/2016
Como Citar
RIBEIRO, Victor V.; TRAVASSOS, Guilherme H..
Testing Non-Functional Requirements: Lacking of Technologies or Researching Opportunities?. In: SIMPÓSIO BRASILEIRO DE QUALIDADE DE SOFTWARE (SBQS), 15. , 2016, Maceió.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2016
.
p. 226-240.
DOI: https://doi.org/10.5753/sbqs.2016.15137.
