Evaluation of the interaction of ubiquitous systems: Applying the MALTU model and Extended Maltu in a IoT water device
Abstract
Ubiquitous systems integrate into people’s daily lives imperceptibly, making it crucial to evaluate these devices to understand users’ needs. This article aims to report the experience of evaluating a smart device using the HCI techniques MALTU and Extended MALTU. To evaluate the device in question, spontaneous user posts (SUPs) were manually and automatically extracted from social platforms, and after extraction, all SUPs were categorized. Using the MALTU method, 41 comments were analyzed, resulting in 38 criticisms (92.6%) and 5 compliments (12.1%). The comments revealed usability issues such as Effectiveness (46.3%), Efficiency (29.2%), Utility (26.8%), Memorability (4.8%), and Satisfaction (2.4%). The Extended MALTU method analyzed 237 comments, with 170 criticisms (63.9%) and 57 compliments (21.4%). The comments revealed aspects related to Context Sensitivity (18.5%), Invisibility (2.5%), Attention (17.5%), Calm (5.9%), Mobility (2.1%), and Synchronicity (53.5%). When observing the results, the MALTU and Extended MALTU models helped identify usability and user experience problems; however, one specifically helped to more explicitly identify issues related to quality criteria in intelligent systems. From the evaluation process using both methods, we highlight in the article some lessons learned and recommendations for evaluating ubiquitous systems.
References
Simone Barbosa and Bruno Silva. 2010. Interação humano-computador. Elsevier Brasil
David Benyon. 2011. Interação humano-computador. Traduçao de Heloısa Coimbra de Souza. 2a. ed. Sao Paulo: Person Prentice Hall (2011), 464.
Rainara Maia Carvalho, Rossana Maria de Castro Andrade, and Káthia Marçal de Oliveira. 2018. AQUArIUM-A suite of software measures for HCI quality evaluation of ubiquitous mobile applications. Journal of Systems and Software 136 (2018), 101–136.
Regina Borges de Araujo. 2003. Computação ubíqua: Princípios, tecnologias e desafios. In XXI Simpósio Brasileiro de Redes de Computadores, Vol. 8. 11–13.
José Cezar de Souza Filho, Marcos Randel Freitas Brito, Antonio Raimundo Rocha Mendonça, Marcos Vitor Martins, and Andréia Libório Sampaio. 2018. Hidrate spark: Avaliando um sistema ubíquo para motivar a ingestão de Água. In Anais Estendidos do XVII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais. SBC.
José Cezar de Souza Filho, Marcos Randel Freitas Brito, and Andréia Libório Sampaio. 2020. Comparing heuristic evaluation and MALTU model in interaction evaluation of ubiquitous systems. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’20). Association for Computing Machinery, New York, NY, USA, Article 10, 10 pages. DOI: 10.1145/3424953.3426639
José Cezar de Souza Filho, Andréia Libório Sampaio, and Eduardo Luzeiro Feitosa. 2022. Hidden things and implicit interaction: looking back on how invisibility evolved as a quality-in-use attribute. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’22). Association for Computing Machinery, New York, NY, USA, Article 19, 11 pages. DOI: 10.1145/3554364.3559136
Carlos EA Feitosa, Ana BV Martins, F Diogo Gaspar, Naily Santos, Ingrid T Monteiro, Marcelo Martins da Silva, et al. 2023. Avaliaçao da Usabilidade e Experiência do Usuário do kit Plantinha IoT. In Anais Estendidos do XXII Simpósio Brasileiro de Fatores Humanos em Sistemas Computacionais. SBC, 51–61.
Carlos Eduardo Almeida Feitosa. 2022. Estendendo o modelo maltu para a avaliação da usabilidade e experiência do usuário em sistemas ubíquos. (2022).
Jutta Fortmann, Vanessa Cobus, Wilko Heuten, and Susanne Boll. 2014. WaterJewel: design and evaluation of a bracelet to promote a better drinking behaviour. In Proceedings of the 13th International Conference on Mobile and Ubiquitous Multimedia (Melbourne, Victoria, Australia) (MUM ’14). Association for Computing Machinery, New York, NY, USA, 58–67. DOI: 10.1145/2677972.2677976
Wagner Ideali. 2021. Conectividade em Automação e IoT: Protocolos I2C, SPI, USB, TCP-IP entre outros. Funcionalidade e interligação para automação e ToT. Alta Books.
Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research methods in human-computer interaction. Morgan Kaufmann.
Luís Gustavo Maschietto, Anderson Luiz Nogueira Vieira, Fernando Esquírio Torres, et al. 2021. Arquitetura e Infraestrutura de IoT. Porto Alegre: Grupo A Educação SA (2021).
Marília Soares Mendes. 2015. MALTU–Um modelo para avaliação da interação em sistemas sociais a partir da linguagem textual do usuário. (2015).
Francisco Moreira and Emanuel Coutinho. 2024. Evaluating the User Experience of Type 1 Diabetes Control Applications (IHC ’23). Association for Computing Machinery, New York, NY, USA, Article 17, 11 pages. DOI: 10.1145/3638067.3638134
Jakob Nielsen. 1994. Usability engineering. Morgan Kaufmann.
Stefan Poslad. 2011. Ubiquitous computing: smart devices, environments and interactions. John Wiley & Sons.
Rainara Maia Santos. 2014. Características e medidas de software para avaliação da qualidade da interação humano-computador em sistemas ubíquos. (2014).
Pedro Victor Sousa Silva, Ruan Gabriel Lopes, Larah Virgínia Pedrosa Lima Cruz, Anna Julia Abreu Lima de Souza, Andréia Libório Sampaio, Marcelo Martins da Silva, et al. 2023. Hidrate Spark TAP: Avaliação de uma Garrafa Inteligente para Motivação e Acompanhamento da Ingestão de Água. In Anais Estendidos do XXII Simpósio Brasileiro de Fatores Humanos em Sistemas Computacionais. SBC, 62–72.
Mark Weiser. 1991. The Computer for the 21 st Century. Scientific american 265, 3 (1991), 94–105.
