A Comprehensive Exploration of the Use of Software Testing Tools for IoT Systems
Resumo
São diversas as ferramentas propostas para a execução de testes de software em sistemas de IoT. No entanto, essas ferramentas são frequentemente pouco acessíveis para uso prático. Existe uma carência de ferramentas que ofereçam suporte robusto para uma estruturação abrangente de testes de software nesses sistemas. Este estudo avalia as ferramentas disponíveis para suporte a testes de software em sistemas de IoT na literatura científica, investigando a viabilidade de utilizá-las em contextos diferentes dos originalmente propostos. Isso é realizado por meio da exploração de ferramentas identificadas em um mapeamento sistemático prévio, além da busca por artefatos e recursos que possam orientar desenvolvedores e testadores em sua utilização. O principal resultado deste artigo consiste na apresentação de evidências sobre as qualidades e limitações das ferramentas de suporte à atividade de teste. Isso proporciona aos programadores e testadores que lidam com aplicações IoT um entendimento mais aprofundado da área e das ferramentas disponíveis.Referências
Ahmed, B. S., Bures, M., Frajtak, K., and Cerny, T. (2019). Aspects of quality in internet of things (iot) solutions: A systematic mapping study. IEEE Access, 7:13758–13780.
Ammann, P. and Offutt, J. (2016). Introduction to software testing. Cambridge University Press, Cambridge, UK.
Bosmans, S., Mercelis, S., Denil, J., and Hellinckx, P. (2019). Testing iot systems using a hybrid simulation based testing approach. Computing, 101:857–872.
Bures, M., Ahmed, B. S., Rechtberger, V., Klima, M., Trnka, M., Jaros, M., Bellekens, X., Almog, D., and Herout, P. (2021). Patriot: Iot automated interoperability and integration testing framework. In 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), pages 454–459, Porto de Galinhas, Brazil. IEEE, IEEE.
Bures, M., Cerny, T., and Ahmed, B. S. (2018). Internet of things: Current challenges in the quality assurance and testing methods. In 9th iCatse Conference on Information Science and Applications, pages 625–634, Hong Kong, China. Springer.
Bures, M., Klima, M., Rechtberger, V., Bellekens, X., Tachtatzis, C., Atkinson, R., and Ahmed, B. S. (2020). Interoperability and integration testing methods for iot systems: A systematic mapping study. In de Boer, F. and Cerone, A., editors, Software Engineering and Formal Methods, pages 93–112, Cham. Springer International Publishing.
Chernyshev, M., Baig, Z., Bello, O., and Zeadally, S. (2018). Internet of things (iot): Research, simulators, and testbeds. IEEE Internet of Things Journal, 5(3):1637–1647.
Clerissi, D., Leotta, M., Reggio, G., and Ricca, F. (2018). Towards an approach for developing and testing node-red iot systems. In Proceedings of the 1st ACM SIGSOFT International Workshop on Ensemble-Based Software Engineering, pages 1–8, Lake Buena Vista, FL, USA. ACM.
Cristea, R., Feraru, M., and Paduraru, C. (2022). Building blocks for iot testing - a benchmark of iot apps and a functional testing framework. In 2022 IEEE/ACM 4th International Workshop on Software Engineering Research and Practices for the IoT (SERP4IoT), pages 25–32, New York, NY, USA. IEEE.
Defalque, G., Santos, R., Pache, M., and Defalque, C. (2023). A review on beef cattle supplementation technologies. Information Processing in Agriculture.
Delamaro, M., Jino, M., and Maldonado, J. (2013). Introdução ao teste de software. Elsevier Brasil, Rio de Janeiro, RJ.
Demirel, S. T., Demirel, M., Dogru, I., and Das, R. (2019). Interopt: A new testing platform based on onem2m standards for iot systems. In 2019 International Symposium on Networks, Computers and Communications (ISNCC), pages 1–6, Istanbul, Turkey. IEEE, IEEE.
Estebsari, A., Patti, E., and Barbierato, L. (2018). Fault detection, isolation and restoration test platform based on smart grid architecture model using intenet-of-things approaches. In 2018 IEEE International Conference on Environment and Electrical Engineering and 2018 IEEE Industrial and Commercial Power Systems Europe (EEEIC / I&CPS Europe), pages 1–5, Palermo, Italy. IEEE.
Ferreira, V. G., Herrera, C. G. a., Souza, S., Santos, R. R. d., and Souza, P. S. L. d. (2023). Software testing applied to the development of iot systems: Preliminary results. In Proceedings of the 8th Brazilian Symposium on Systematic and Automated Software Testing, SAST ’23, page 113–122, New York, NY, USA. Association for Computing Machinery.
Gartner (2020). Gartner forecasts worldwide iot endpoint electronics revenue to reach $389 billion in 2030.
Han, S. N., Lee, G. M., Crespi, N., Heo, K., Van Luong, N., Brut, M., and Gatellier, P. (2014). Dpwsim: A simulation toolkit for iot applications using devices profile for web services. In 2014 IEEE World Forum on Internet of Things (WF-IoT), pages 544–547, Seoul, Korea. IEEE, IEEE.
Haris, I., Bisanovic, V., Wally, B., Rausch, T., Ratasich, D., Dustdar, S., Kappel, G., and Grosu, R. (2019). Sensyml: Simulation environment for large-scale iot applications. In IECON 2019 - 45th Annual Conference of the IEEE Industrial Electronics Society, page 3024–3030, Lisbon, Portugal. IEEE Press.
Jha, D. N., Alwasel, K., Alshoshan, A., Huang, X., Naha, R. K., Battula, S. K., Garg, S., Puthal, D., James, P., Zomaya, A., et al. (2020). Iotsim-edge: a simulation framework for modeling the behavior of internet of things and edge computing environments. Software: Practice and Experience, 50(6):844–867.
Klima, M., Bures, M., Ahmed, B. S., Bellekens, X., Atkinson, R., Tachtatzis, C., and Herout, P. (2023). Specialized path-based technique to test internet of things system functionality under limited network connectivity. Internet of Things, 22:100706.
Kuroiwa, T., Aoyama, Y., and Kushiro, N. (2019). A hybrid testing environment between execution test and model checking for iot system. In 2019 IEEE International Conference on Consumer Electronics (ICCE), pages 1–2, Las Vegas, USA. IEEE, IEEE.
Li, J., Nejati, S., Sabetzadeh, M., and McCallen, M. (2022). A domain-specific language for simulation-based testing of iot edge-to-cloud solutions. In Proceedings of the 25th International Conference on Model Driven Engineering Languages and Systems, pages 367–378, New York, NY, USA. Association for Computing Machinery.
Minani, J. B., Sabir, F., Moha, N., and Guéhéneuc, Y.-G. (2024). A multimethod study of internet of things systems testing in industry. IEEE Internet of Things Journal, 11(1):1662–1684.
Murad, G., Badarneh, A., Qusef, A., and Almasalha, F. (2018). Software testing techniques in iot. In 2018 8th International Conference on Computer Science and Information Technology (CSIT), pages 17–21.
Patel, D., Mehtre, M., and Wankar, R. (2019). Simulators, emulators, and test-beds for internet of things: A comparison. In 2019 Third International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), pages 139–145, Palladam, India. IEEE, IEEE.
Pflanzner, T., Kertesz, A., Spinnewyn, B., and Latré, S. (2016). Mobiotsim: Towards a mobile iot device simulator. In 2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), pages 21–27, Vienna, Austria. IEEE.
Saraiva, R., Mello, P., Soares, P., Souza, M., and Cortés, M. (2019). Adoption of software testing in internet of things: A systematic literature mapping. In Proceedings of the IV Brazilian Symposium on Systematic and Automated Software Testing, pages 3–11, Salvador, Brazil. ACM, ACM.
Schieferdecker, I., Kretzschmann, S., Rennoch, A., and Wagner, M. (2017). Iot-testware - an eclipse project. In 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS), pages 1–8, Prague, Czech Republic. IEEE.
Varghese, N. and Sinha, R. (2020). Can commercial testing automation tools work for iot? a case study of selenium and node-red. In IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society, pages 4519–4524, Singapore. IEEE, IEEE.
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th international conference on evaluation and assessment in software engineering, pages 1–10, London, England, United Kingdom. ACM, ACM.
Wu, M., Lu, T.-J., Ling, F.-Y., Sun, J., and Du, H.-Y. (2010). Research on the architecture of internet of things. In 2010 3rd International Conference on Advanced Computer Theory and Engineering(ICACTE), volume 5, pages V5–484–V5–487.
Ammann, P. and Offutt, J. (2016). Introduction to software testing. Cambridge University Press, Cambridge, UK.
Bosmans, S., Mercelis, S., Denil, J., and Hellinckx, P. (2019). Testing iot systems using a hybrid simulation based testing approach. Computing, 101:857–872.
Bures, M., Ahmed, B. S., Rechtberger, V., Klima, M., Trnka, M., Jaros, M., Bellekens, X., Almog, D., and Herout, P. (2021). Patriot: Iot automated interoperability and integration testing framework. In 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), pages 454–459, Porto de Galinhas, Brazil. IEEE, IEEE.
Bures, M., Cerny, T., and Ahmed, B. S. (2018). Internet of things: Current challenges in the quality assurance and testing methods. In 9th iCatse Conference on Information Science and Applications, pages 625–634, Hong Kong, China. Springer.
Bures, M., Klima, M., Rechtberger, V., Bellekens, X., Tachtatzis, C., Atkinson, R., and Ahmed, B. S. (2020). Interoperability and integration testing methods for iot systems: A systematic mapping study. In de Boer, F. and Cerone, A., editors, Software Engineering and Formal Methods, pages 93–112, Cham. Springer International Publishing.
Chernyshev, M., Baig, Z., Bello, O., and Zeadally, S. (2018). Internet of things (iot): Research, simulators, and testbeds. IEEE Internet of Things Journal, 5(3):1637–1647.
Clerissi, D., Leotta, M., Reggio, G., and Ricca, F. (2018). Towards an approach for developing and testing node-red iot systems. In Proceedings of the 1st ACM SIGSOFT International Workshop on Ensemble-Based Software Engineering, pages 1–8, Lake Buena Vista, FL, USA. ACM.
Cristea, R., Feraru, M., and Paduraru, C. (2022). Building blocks for iot testing - a benchmark of iot apps and a functional testing framework. In 2022 IEEE/ACM 4th International Workshop on Software Engineering Research and Practices for the IoT (SERP4IoT), pages 25–32, New York, NY, USA. IEEE.
Defalque, G., Santos, R., Pache, M., and Defalque, C. (2023). A review on beef cattle supplementation technologies. Information Processing in Agriculture.
Delamaro, M., Jino, M., and Maldonado, J. (2013). Introdução ao teste de software. Elsevier Brasil, Rio de Janeiro, RJ.
Demirel, S. T., Demirel, M., Dogru, I., and Das, R. (2019). Interopt: A new testing platform based on onem2m standards for iot systems. In 2019 International Symposium on Networks, Computers and Communications (ISNCC), pages 1–6, Istanbul, Turkey. IEEE, IEEE.
Estebsari, A., Patti, E., and Barbierato, L. (2018). Fault detection, isolation and restoration test platform based on smart grid architecture model using intenet-of-things approaches. In 2018 IEEE International Conference on Environment and Electrical Engineering and 2018 IEEE Industrial and Commercial Power Systems Europe (EEEIC / I&CPS Europe), pages 1–5, Palermo, Italy. IEEE.
Ferreira, V. G., Herrera, C. G. a., Souza, S., Santos, R. R. d., and Souza, P. S. L. d. (2023). Software testing applied to the development of iot systems: Preliminary results. In Proceedings of the 8th Brazilian Symposium on Systematic and Automated Software Testing, SAST ’23, page 113–122, New York, NY, USA. Association for Computing Machinery.
Gartner (2020). Gartner forecasts worldwide iot endpoint electronics revenue to reach $389 billion in 2030.
Han, S. N., Lee, G. M., Crespi, N., Heo, K., Van Luong, N., Brut, M., and Gatellier, P. (2014). Dpwsim: A simulation toolkit for iot applications using devices profile for web services. In 2014 IEEE World Forum on Internet of Things (WF-IoT), pages 544–547, Seoul, Korea. IEEE, IEEE.
Haris, I., Bisanovic, V., Wally, B., Rausch, T., Ratasich, D., Dustdar, S., Kappel, G., and Grosu, R. (2019). Sensyml: Simulation environment for large-scale iot applications. In IECON 2019 - 45th Annual Conference of the IEEE Industrial Electronics Society, page 3024–3030, Lisbon, Portugal. IEEE Press.
Jha, D. N., Alwasel, K., Alshoshan, A., Huang, X., Naha, R. K., Battula, S. K., Garg, S., Puthal, D., James, P., Zomaya, A., et al. (2020). Iotsim-edge: a simulation framework for modeling the behavior of internet of things and edge computing environments. Software: Practice and Experience, 50(6):844–867.
Klima, M., Bures, M., Ahmed, B. S., Bellekens, X., Atkinson, R., Tachtatzis, C., and Herout, P. (2023). Specialized path-based technique to test internet of things system functionality under limited network connectivity. Internet of Things, 22:100706.
Kuroiwa, T., Aoyama, Y., and Kushiro, N. (2019). A hybrid testing environment between execution test and model checking for iot system. In 2019 IEEE International Conference on Consumer Electronics (ICCE), pages 1–2, Las Vegas, USA. IEEE, IEEE.
Li, J., Nejati, S., Sabetzadeh, M., and McCallen, M. (2022). A domain-specific language for simulation-based testing of iot edge-to-cloud solutions. In Proceedings of the 25th International Conference on Model Driven Engineering Languages and Systems, pages 367–378, New York, NY, USA. Association for Computing Machinery.
Minani, J. B., Sabir, F., Moha, N., and Guéhéneuc, Y.-G. (2024). A multimethod study of internet of things systems testing in industry. IEEE Internet of Things Journal, 11(1):1662–1684.
Murad, G., Badarneh, A., Qusef, A., and Almasalha, F. (2018). Software testing techniques in iot. In 2018 8th International Conference on Computer Science and Information Technology (CSIT), pages 17–21.
Patel, D., Mehtre, M., and Wankar, R. (2019). Simulators, emulators, and test-beds for internet of things: A comparison. In 2019 Third International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), pages 139–145, Palladam, India. IEEE, IEEE.
Pflanzner, T., Kertesz, A., Spinnewyn, B., and Latré, S. (2016). Mobiotsim: Towards a mobile iot device simulator. In 2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), pages 21–27, Vienna, Austria. IEEE.
Saraiva, R., Mello, P., Soares, P., Souza, M., and Cortés, M. (2019). Adoption of software testing in internet of things: A systematic literature mapping. In Proceedings of the IV Brazilian Symposium on Systematic and Automated Software Testing, pages 3–11, Salvador, Brazil. ACM, ACM.
Schieferdecker, I., Kretzschmann, S., Rennoch, A., and Wagner, M. (2017). Iot-testware - an eclipse project. In 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS), pages 1–8, Prague, Czech Republic. IEEE.
Varghese, N. and Sinha, R. (2020). Can commercial testing automation tools work for iot? a case study of selenium and node-red. In IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society, pages 4519–4524, Singapore. IEEE, IEEE.
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th international conference on evaluation and assessment in software engineering, pages 1–10, London, England, United Kingdom. ACM, ACM.
Wu, M., Lu, T.-J., Ling, F.-Y., Sun, J., and Du, H.-Y. (2010). Research on the architecture of internet of things. In 2010 3rd International Conference on Advanced Computer Theory and Engineering(ICACTE), volume 5, pages V5–484–V5–487.
Publicado
24/05/2024
Como Citar
HERRERA, Caio Guimarães; FERREIRA, Vinicius Gomes; SOUZA, Simone R. S.; SANTOS, Ricardo; SOUZA, Paulo Sergio Lopes de.
A Comprehensive Exploration of the Use of Software Testing Tools for IoT Systems. In: WORKSHOP DE TESTES E TOLERÂNCIA A FALHAS (WTF), 25. , 2024, Niterói/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 57-70.
ISSN 2595-2684.
DOI: https://doi.org/10.5753/wtf.2024.3247.
