Quality Evaluation of Self-Adaptive Systems: Challenges and Opportunities
Resumo
Self-adaptive systems (SAS) can adapt their behavior to suit user preferences or contexts, as well as monitor their performance and adjust it if necessary. In addition to adaptation operations, self-adaptive systems communicate with sensors, actuators, and other devices. Due to the complexity and dynamism of SAS, many situations can compromise the functioning of the system, such as faults in adaptations, low performance to execute tasks, and context inconsistencies. To prevent the system of these problems, it is essential to ensure high levels of quality. However, due to the peculiarities of these systems, there are still challenges to perform quality evaluations in these systems. In this sense, this paper proposes a discussion about the quality evaluation of self-adaptive systems in the last years. As a result, we identify challenges, limitations and research opportunities related to SAS quality evaluation.
Referências
S. Adjoyan and A. Serial. 2017. Reconfigurable service-based architecture based on variability description. In Proceedings of the Symposium on Applied Computing. ACM, 1154--1161.
C. I. M. Bezerra, R. M. C. Andrade, J. Monteiro, and D. Cedraz. 2018. Aggregating Measures using Fuzzy Logic for Evaluating Feature Models. In Proceedings of the 12th International Workshop on Variability Modelling of Software-Intensive Systems. ACM, 35--42.
C. I. M. Bezerra, J. Barbosa, J. H. Freires, R. M. C. Andrade, and J. Monteiro. 2016. DyMMer: a measurement-based tool to support quality evaluation of DSPL feature models. In Proceedings of the 20th International Systems and Software Product Line Conference. ACM, 314--317.
Y. Brun, G. Di Marzo Serugendo, C. Gacek, H. Giese, H. Kienle, M. Litoiu, H. Müller, M. Pezzè, and M. Shaw. 2009. Engineering Self-Adaptive Systems through Feedback Loops. Springer Berlin Heidelberg, Berlin, Heidelberg, 48--70. https: //doi.org/10.1007/978-3-642-02161-9_3
M. Camilli, C. Bellettini, A. Gargantini, and P. Scandurra. 2018. Online Model-Based Testing under Uncertainty. In 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE). 36--46. https://doi.org/10.1109/ISSRE.2018.00015
C. Cetina, P. Giner, J. Fons, and V. Pelechano. 2013. Prototyping Dynamic Software Product Lines to evaluate run-time reconfigurations. Science of Computer Programming 78, 12 (2013), 2399--2413. https://doi.org/10.1016/j.scico.2012.06.007 Special Section on International Software Product Line Conference 2010 and Fundamentals of Software Engineering (selected papers of FSEN 2011).
B. Chen, X. Peng, Y. Liu, S. Song, J. Zheng, and W. Zhao. 2016. Architecture-based behavioral adaptation with generated alternatives and relaxed constraints. IEEE Transactions on Services Computing (2016).
B. H. C. Cheng, R. de Lemos, H. Giese, P. Inverardi, J. Magee, J. Andersson, B. Becker, N. Bencomo, Y. Brun, B. Cukic, G. Di Marzo Serugendo, S. Dustdar, A. Finkelstein, C. Gacek, K. Geihs, V. Grassi, G. Karsai, H. M. Kienle, J. Kramer, M. Litoiu, S. Malek, R. Mirandola, H. A. Müller, S. Park, M. Shaw, M. Tichy, M. Tivoli, D. Weyns, and J. Whittle. 2009. Software Engineering for Self-Adaptive Systems: A Research Roadmap. Springer Berlin Heidelberg, Berlin, Heidelberg, 1--26. https://doi.org/10.1007/978-3-642-02161-9_1
J. Criado, S. Martínez-Fernández, D. Ameller, L. Iribarne, and N. Padilla. 2016. Exploring Quality-Aware Architectural Transformations at Run-Time: The ENIA Case. In Model and Data Engineering, Ladjel Bellatreche, Óscar Pastor, Jesús M. Almendros Jiménez, and Yamine Aït-Ameur (Eds.). Springer International Publishing, Cham, 288--302.
J. Criado, S. Martínez-Fernández, D. Ameller, L. Iribarne, N. Padilla, and A. Jedlitschka. 2018. Quality-aware architectural model transformations in adaptive mashups user interfaces. Fundamenta Informaticae 162, 4 (2018), 283--309.
R. Edwards and N. Bencomo. 2018. DeSiRE: Further Understanding Nuances of Degrees of Satisfaction of Non-functional Requirements Trade-off. In Proceedings of the 13th International Conference on Software Engineering for Adaptive and Self-Managing Systems (SEAMS '18). ACM, New York, NY, USA, 12--18. https://doi.org/10.1145/3194133.3194142
A. Farahani, E. Nazemi, G. Cabri, and A. Rafizadeh. 2017. An evaluation method for Self-Adaptive systems. 2016 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2016 - Conference Proceedings (2017), 2814--2820. https: //doi.org/10.1109/SMC.2016.7844665 cited By 0.
J. M. Franco, F. Correia, R. Barbosa, M. Zenha-Rela, B. Schmerl, and D. Garlan. 2016. Improving self-adaptation planning through software architecture-based stochastic modeling. Journal of Systems and Software 115 (2016), 42--60. https://doi.org/10.1016/j.jss.2016.01.026
A. Goldsteen, M. Moffie, T. Bandyszak, N.G. Mohammadi, X. Chen, S. Meichanetzoglou, S. Ioannidis, and P. Chatziadam. 2015. A tool for monitoring and maintaining system trustworthiness at runtime. CEUR Workshop Proceedings 1342 (2015), 142--147. cited By 3.
A. Imed and M. Graiet. 2017. An automatic configuration algorithm for reliable and efficient composite services. IEEE Transactions on Network and Service Management 15, 1 (2017), 416--429.
ISO. 2011. IEC 25010: 2011 systems and software engineering--systems and software quality requirements and evaluation (square)--system and software quality models. International Organization for Standardization 34 (2011), 2910.
E. Kaddoum, C. Raibulet, J. Georgé,G. Picard,and M. Gleizes. 2010. Criteria for the Evaluation of Self-Systems. In Proceedings of the 2010 ICSE Workshop on Software Engineering for Adaptive and Self-Managing Systems (SEAMS '10). ACM, New York, NY, USA, 29--38. https://doi.org/10.1145/1808984.1808988
R. Laddaga and P. Robertson. 2004. Self adaptive software: A position paper. In SELF-STAR: International Workshop on Self- Properties in Complex Information Systems, Vol. 31. 19.
R. Moein Far and A. A. Barforoush. 2017. Using models at run-time to measure quality of SAS in the large-scale software systems. In 2017 9th International Conference on Information and Knowledge Technology (IKT). IEEE, 99--103.
F. A. Moghaddam, G. Procaccianti, G. A. Lewis, and P. Lago. 2018. Empirical validation of cyber-foraging architectural tactics for surrogate provisioning. Journal of Systems and Software 138 (2018), 37--51.
S. Neti and H. A. Muller. 2007. Quality Criteria and an Analysis Framework for Self-Healing Systems. In International Workshop on Software Engineering for Adaptive and Self-Managing Systems (SEAMS '07). 6--6. https://doi.org/10.1109/SEAMS.2007.15
S. Park, S. Park, and K. Ma. 2018. An Automatic User Activity Analysis Method for Discovering Latent Requirements: Usability Issue Detection on Mobile Applications. Sensors 18, 9 (2018), 2963.
L.H.G. Paucar and N. Bencomo. 2017. The reassessment of preferences of non-functional requirements for better informed decision-making in self-adaptation. Proceedings - 2016 IEEE 24th International Requirements Engineering Conference Workshops, REW 2016 (2017), 32--38. https://doi.org/10.1109/REW.2016.38 cited By 5.
L. Pessoa, P. Fernandes, T. Castro, V. Alves, G. N. Rodrigues, and H. Carvalho. 2017. Building reliable and maintainable Dynamic Software Product Lines: An investigation in the Body Sensor Network domain. Information and Software Technology 86 (2017), 54--70. https://doi.org/10.1016/j.infsof.2017.02.002
P. Potena. 2013. Optimization of adaptation plans for a service-oriented architecture with cost, reliability, availability and performance tradeoff. Journal of Systems and Software 86, 3 (2013), 624--648. https://doi.org/10.1016/j.jss.2012.10.929
C. Raibulet. 2014. Hints on Quality Evaluation of Self-Systems. In 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems. 185--186. https://doi.org/10.1109/SASO.2014.36
C. Raibulet, F. Arcelli Fontana, R. Capilla, and C. Carrillo. 2016. An Overview on Quality Evaluation of Self-Adaptive Systems. Managing Trade-offs in Adaptable Software Architectures (2016).
C. Raibulet, F. Arcelli Fontana, R. Capilla, and C. Carrillo. 2017. Chapter 13 - An Overview on Quality Evaluation of Self-Adaptive Systems. In Managing Trade-Offs in Adaptable Software Architectures, Ivan Mistrik, Nour Ali, Rick Kazman, John Grundy, and Bradley Schmerl (Eds.). Morgan Kaufmann, Boston, 325--352. https://doi.org/10.1016/B978-0-12-802855-1.00013-7
A. Rodrigues, R. D. Caldas, G. N. Rodrigues, T. Vogel, and P. Pelliccione. 2018. A learning approach to enhance assurances for real-time self-adaptive systems. In 2018 IEEE/ACM 13th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS). IEEE, 206--216.
L. Sabatucci, V. Seidita, and M. Cossentino. 2018. The four types of self-adaptive systems: A metamodel. Smart Innovation, Systems and Technologies 76 (2018), 440--450. https://doi.org/10.1007/978-3-319-59480-4_44 cited By 1.
A. A. A. Saeed and S. Lee. 2018. Non-functional Requirements Trade-Off in Self-Adaptive Systems. In 2018 4th International Workshop on Requirements Engineering for Self-Adaptive, Collaborative, and Cyber Physical Systems (RESACS). IEEE, 9--15.
M. Salehie and L. Tahvildari. 2009. Self-adaptive software: Landscape and research challenges. ACM transactions on autonomous and adaptive systems (TAAS) 4, 2 (2009), 14.
L. E. Sanchez, J. A. Diaz-Pace, A. Zunino, S. Moisan, and J. Rigault. 2015. An approach based on feature models and quality criteria for adapting component-based systems. Journal of Software Engineering Research and Development 3, 1 (23 Jun 2015), 10. https://doi.org/10.1186/s40411-015-0022-1
T. Sanislav, G. Mois, and L. Miclea. 2016. An approach to model dependability of cyber-physical systems. Microprocessors and Microsystems 41 (2016), 67--76. https://doi.org/10.1016/j.micpro.2015.11.021
E. Serral, P. Sernani, and F. Dalpiaz. 2018. Personalized adaptation in pervasive systems via non-functional requirements. Journal of Ambient Intelligence and Humanized Computing 9, 6 (01 Nov 2018), 1729--1743. https://doi.org/10.1007/s12652-017-0611-4
P. Smiari and S. Bibi. 2018. A Smart City Application Modeling Framework: A Case Study on Re-engineering a Smart Retail Platform. In 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE, 111--118.
B.Vogel-Heuser and J. Prieler. 2017. Evaluation of selected metrics for flexibility of Cyber Physical Production Systems. In 2017 13th IEEE Conference on Automation Science and Engineering(CASE). IEEE, 701--708.
J. Yang, G. Huang, W. Zhu, X. Cui, and H. Mei. 2009. Quality attribute tradeoff through adaptive architectures at runtime. Journal of Systems and Software 82, 2 (2009), 319--332. https://doi.org/10.1016/j.jss.2008.06.039
C. I. M. Bezerra, R. M. C. Andrade, J. Monteiro, and D. Cedraz. 2018. Aggregating Measures using Fuzzy Logic for Evaluating Feature Models. In Proceedings of the 12th International Workshop on Variability Modelling of Software-Intensive Systems. ACM, 35--42.
C. I. M. Bezerra, J. Barbosa, J. H. Freires, R. M. C. Andrade, and J. Monteiro. 2016. DyMMer: a measurement-based tool to support quality evaluation of DSPL feature models. In Proceedings of the 20th International Systems and Software Product Line Conference. ACM, 314--317.
Y. Brun, G. Di Marzo Serugendo, C. Gacek, H. Giese, H. Kienle, M. Litoiu, H. Müller, M. Pezzè, and M. Shaw. 2009. Engineering Self-Adaptive Systems through Feedback Loops. Springer Berlin Heidelberg, Berlin, Heidelberg, 48--70. https: //doi.org/10.1007/978-3-642-02161-9_3
M. Camilli, C. Bellettini, A. Gargantini, and P. Scandurra. 2018. Online Model-Based Testing under Uncertainty. In 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE). 36--46. https://doi.org/10.1109/ISSRE.2018.00015
C. Cetina, P. Giner, J. Fons, and V. Pelechano. 2013. Prototyping Dynamic Software Product Lines to evaluate run-time reconfigurations. Science of Computer Programming 78, 12 (2013), 2399--2413. https://doi.org/10.1016/j.scico.2012.06.007 Special Section on International Software Product Line Conference 2010 and Fundamentals of Software Engineering (selected papers of FSEN 2011).
B. Chen, X. Peng, Y. Liu, S. Song, J. Zheng, and W. Zhao. 2016. Architecture-based behavioral adaptation with generated alternatives and relaxed constraints. IEEE Transactions on Services Computing (2016).
B. H. C. Cheng, R. de Lemos, H. Giese, P. Inverardi, J. Magee, J. Andersson, B. Becker, N. Bencomo, Y. Brun, B. Cukic, G. Di Marzo Serugendo, S. Dustdar, A. Finkelstein, C. Gacek, K. Geihs, V. Grassi, G. Karsai, H. M. Kienle, J. Kramer, M. Litoiu, S. Malek, R. Mirandola, H. A. Müller, S. Park, M. Shaw, M. Tichy, M. Tivoli, D. Weyns, and J. Whittle. 2009. Software Engineering for Self-Adaptive Systems: A Research Roadmap. Springer Berlin Heidelberg, Berlin, Heidelberg, 1--26. https://doi.org/10.1007/978-3-642-02161-9_1
J. Criado, S. Martínez-Fernández, D. Ameller, L. Iribarne, and N. Padilla. 2016. Exploring Quality-Aware Architectural Transformations at Run-Time: The ENIA Case. In Model and Data Engineering, Ladjel Bellatreche, Óscar Pastor, Jesús M. Almendros Jiménez, and Yamine Aït-Ameur (Eds.). Springer International Publishing, Cham, 288--302.
J. Criado, S. Martínez-Fernández, D. Ameller, L. Iribarne, N. Padilla, and A. Jedlitschka. 2018. Quality-aware architectural model transformations in adaptive mashups user interfaces. Fundamenta Informaticae 162, 4 (2018), 283--309.
R. Edwards and N. Bencomo. 2018. DeSiRE: Further Understanding Nuances of Degrees of Satisfaction of Non-functional Requirements Trade-off. In Proceedings of the 13th International Conference on Software Engineering for Adaptive and Self-Managing Systems (SEAMS '18). ACM, New York, NY, USA, 12--18. https://doi.org/10.1145/3194133.3194142
A. Farahani, E. Nazemi, G. Cabri, and A. Rafizadeh. 2017. An evaluation method for Self-Adaptive systems. 2016 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2016 - Conference Proceedings (2017), 2814--2820. https: //doi.org/10.1109/SMC.2016.7844665 cited By 0.
J. M. Franco, F. Correia, R. Barbosa, M. Zenha-Rela, B. Schmerl, and D. Garlan. 2016. Improving self-adaptation planning through software architecture-based stochastic modeling. Journal of Systems and Software 115 (2016), 42--60. https://doi.org/10.1016/j.jss.2016.01.026
A. Goldsteen, M. Moffie, T. Bandyszak, N.G. Mohammadi, X. Chen, S. Meichanetzoglou, S. Ioannidis, and P. Chatziadam. 2015. A tool for monitoring and maintaining system trustworthiness at runtime. CEUR Workshop Proceedings 1342 (2015), 142--147. cited By 3.
A. Imed and M. Graiet. 2017. An automatic configuration algorithm for reliable and efficient composite services. IEEE Transactions on Network and Service Management 15, 1 (2017), 416--429.
ISO. 2011. IEC 25010: 2011 systems and software engineering--systems and software quality requirements and evaluation (square)--system and software quality models. International Organization for Standardization 34 (2011), 2910.
E. Kaddoum, C. Raibulet, J. Georgé,G. Picard,and M. Gleizes. 2010. Criteria for the Evaluation of Self-Systems. In Proceedings of the 2010 ICSE Workshop on Software Engineering for Adaptive and Self-Managing Systems (SEAMS '10). ACM, New York, NY, USA, 29--38. https://doi.org/10.1145/1808984.1808988
R. Laddaga and P. Robertson. 2004. Self adaptive software: A position paper. In SELF-STAR: International Workshop on Self- Properties in Complex Information Systems, Vol. 31. 19.
R. Moein Far and A. A. Barforoush. 2017. Using models at run-time to measure quality of SAS in the large-scale software systems. In 2017 9th International Conference on Information and Knowledge Technology (IKT). IEEE, 99--103.
F. A. Moghaddam, G. Procaccianti, G. A. Lewis, and P. Lago. 2018. Empirical validation of cyber-foraging architectural tactics for surrogate provisioning. Journal of Systems and Software 138 (2018), 37--51.
S. Neti and H. A. Muller. 2007. Quality Criteria and an Analysis Framework for Self-Healing Systems. In International Workshop on Software Engineering for Adaptive and Self-Managing Systems (SEAMS '07). 6--6. https://doi.org/10.1109/SEAMS.2007.15
S. Park, S. Park, and K. Ma. 2018. An Automatic User Activity Analysis Method for Discovering Latent Requirements: Usability Issue Detection on Mobile Applications. Sensors 18, 9 (2018), 2963.
L.H.G. Paucar and N. Bencomo. 2017. The reassessment of preferences of non-functional requirements for better informed decision-making in self-adaptation. Proceedings - 2016 IEEE 24th International Requirements Engineering Conference Workshops, REW 2016 (2017), 32--38. https://doi.org/10.1109/REW.2016.38 cited By 5.
L. Pessoa, P. Fernandes, T. Castro, V. Alves, G. N. Rodrigues, and H. Carvalho. 2017. Building reliable and maintainable Dynamic Software Product Lines: An investigation in the Body Sensor Network domain. Information and Software Technology 86 (2017), 54--70. https://doi.org/10.1016/j.infsof.2017.02.002
P. Potena. 2013. Optimization of adaptation plans for a service-oriented architecture with cost, reliability, availability and performance tradeoff. Journal of Systems and Software 86, 3 (2013), 624--648. https://doi.org/10.1016/j.jss.2012.10.929
C. Raibulet. 2014. Hints on Quality Evaluation of Self-Systems. In 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems. 185--186. https://doi.org/10.1109/SASO.2014.36
C. Raibulet, F. Arcelli Fontana, R. Capilla, and C. Carrillo. 2016. An Overview on Quality Evaluation of Self-Adaptive Systems. Managing Trade-offs in Adaptable Software Architectures (2016).
C. Raibulet, F. Arcelli Fontana, R. Capilla, and C. Carrillo. 2017. Chapter 13 - An Overview on Quality Evaluation of Self-Adaptive Systems. In Managing Trade-Offs in Adaptable Software Architectures, Ivan Mistrik, Nour Ali, Rick Kazman, John Grundy, and Bradley Schmerl (Eds.). Morgan Kaufmann, Boston, 325--352. https://doi.org/10.1016/B978-0-12-802855-1.00013-7
A. Rodrigues, R. D. Caldas, G. N. Rodrigues, T. Vogel, and P. Pelliccione. 2018. A learning approach to enhance assurances for real-time self-adaptive systems. In 2018 IEEE/ACM 13th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS). IEEE, 206--216.
L. Sabatucci, V. Seidita, and M. Cossentino. 2018. The four types of self-adaptive systems: A metamodel. Smart Innovation, Systems and Technologies 76 (2018), 440--450. https://doi.org/10.1007/978-3-319-59480-4_44 cited By 1.
A. A. A. Saeed and S. Lee. 2018. Non-functional Requirements Trade-Off in Self-Adaptive Systems. In 2018 4th International Workshop on Requirements Engineering for Self-Adaptive, Collaborative, and Cyber Physical Systems (RESACS). IEEE, 9--15.
M. Salehie and L. Tahvildari. 2009. Self-adaptive software: Landscape and research challenges. ACM transactions on autonomous and adaptive systems (TAAS) 4, 2 (2009), 14.
L. E. Sanchez, J. A. Diaz-Pace, A. Zunino, S. Moisan, and J. Rigault. 2015. An approach based on feature models and quality criteria for adapting component-based systems. Journal of Software Engineering Research and Development 3, 1 (23 Jun 2015), 10. https://doi.org/10.1186/s40411-015-0022-1
T. Sanislav, G. Mois, and L. Miclea. 2016. An approach to model dependability of cyber-physical systems. Microprocessors and Microsystems 41 (2016), 67--76. https://doi.org/10.1016/j.micpro.2015.11.021
E. Serral, P. Sernani, and F. Dalpiaz. 2018. Personalized adaptation in pervasive systems via non-functional requirements. Journal of Ambient Intelligence and Humanized Computing 9, 6 (01 Nov 2018), 1729--1743. https://doi.org/10.1007/s12652-017-0611-4
P. Smiari and S. Bibi. 2018. A Smart City Application Modeling Framework: A Case Study on Re-engineering a Smart Retail Platform. In 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE, 111--118.
B.Vogel-Heuser and J. Prieler. 2017. Evaluation of selected metrics for flexibility of Cyber Physical Production Systems. In 2017 13th IEEE Conference on Automation Science and Engineering(CASE). IEEE, 701--708.
J. Yang, G. Huang, W. Zhu, X. Cui, and H. Mei. 2009. Quality attribute tradeoff through adaptive architectures at runtime. Journal of Systems and Software 82, 2 (2009), 319--332. https://doi.org/10.1016/j.jss.2008.06.039
Publicado
21/01/2020
Como Citar
DE SOUSA, Amanda Oliveira; BEZERRA, Carla I. M.; ANDRADE, Rossana M. C.; M. FILHO, José M. S..
Quality Evaluation of Self-Adaptive Systems: Challenges and Opportunities. In: INSIGHTFUL IDEAS AND EMERGING RESULTS - SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE (SBES), 33. , 2019, Salvador.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
