Evaluating a MALTU Extension for UbiComp and IoT Systems
Resumo
In the Ubiquitous Computing (UbiComp) and Internet of Things (IoT) paradigms, systems have certain specific quality-of-use characteristics that traditional Human-Computer Interaction (HCI) evaluation methods may not be able to address, such as context-awareness, invisibility, attentiveness, calmness, mobility and synchronicity, considering that most HCI assessments were generally created to evaluate desktop systems. MALTU model evaluates usability and user experience based on comments left by users on social systems (e.g. Amazom, Mercado Livre, Facebook forums). This work presents an extention in the model to adress these quality characteristics. Before proposing the extension, three evaluations were peformed with the original MALTU. Then, three evaluators performed two evaluations each on the same smart device: the first was the application of the extended MALTU, and the second was a heuristic evaluation using a specific set of heuristics for UbiComp systems. After that, a focus group was held in order to verify if the extended MALTU was capable of identifying typical problems of ubiquitous devices considering its quality characteristics. The results indicate that the extension was able to identify interaction problems in ubiquitous devices, mainly related to usability and user experience, even with difficulties in understanding the concepts of quality characteristics of ubiquitous and IoT systems.
Palavras-chave:
Human-centered computing, Ubiquitous and mobile computing design and evaluation methods, Heuristic evaluations
Referências
Rossana M. C. Andrade, Rainara M. Carvalho, Italo Linhares de Araújo, Káthia M. Oliveira, and Marcio E. F. Maia. 2017. What Changes from Ubiquitous Computing to Internet of Things in Interaction Evaluation?. In Distributed, Ambient and Pervasive Interactions: 5th International Conference, DAPI 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9–14, 2017, Proceedings (Vancouver, BC, Canada). Springer-Verlag, Berlin, Heidelberg, 3–21. DOI: 10.1007/978-3-319-58697-7_1
Parvaneh Asghari, Amir Masoud Rahmani, and Hamid Haj Seyyed Javadi. 2019. Internet of Things applications: A systematic review. Computer Networks 148 (2019), 241–261.
Simone Barbosa and Bruno Silva. 2010. Interação humano-computador. Elsevier Brasil.
Marcos Randel Freitas Brito. 2018. Explorando o modelo MALTU na avaliação de usabilidade e experiência de uso em um dispositivo ubíquo. (2018).
Rainara Maia Carvalho. 2017. Dealing with Conflicts Between Non-functional Requirements of UbiComp and IoT Applications. In 2017 IEEE 25th International Requirements Engineering Conference (RE). 544–549. DOI: 10.1109/RE.2017.51
Rainara Maia Carvalho. 2019. Correlate & lead: process and catalog of non-functional requirements correlations in ubicomp and iot systems. (2019).
Rainara Maia Carvalho, Rossana Maria de Castro Andrade, and Káthia Marçal de Oliveira. 2018. AQUArIUM-A suite of software measures for HCI quality evaluation of ubiquitous mobile applications. Journal of Systems and Software 136 (2018), 101–136.
Rainara Maia Carvalho, Rossana Maria de Castro Andrade, Káthia Marçal de Oliveira, Ismayle de Sousa Santos, and Carla Ilane Moreira Bezerra. 2017. Quality characteristics and measures for human–computer interaction evaluation in ubiquitous systems. Software Quality Journal 25 (2017), 743–795.
Clarisse Sieckenius De Souza. 2005. The semiotic engineering of human-computer interaction. MIT press.
Luiz Gustavo de Souza, Simone Diniz Junqueira Barbosa, and Tayana Conte. 2015. Evaluating a MoLIC Extension for Collaborative Systems Design. In Proceedings of the 14th Brazilian Symposium on Human Factors in Computing Systems (Salvador, Brazil) (IHC ’15). Association for Computing Machinery, New York, NY, USA, Article 15, 10 pages. DOI: 10.1145/3148456.3148471
José Cezar de Souza Filho, Marcos Randel Freitas Brito, Antonio Raimundo Rocha Mendonça, Marcos Vitor Martins, and Andréia Libório Sampaio. 2018. Hidrate Spark: Avaliando um Sistema Ubíquo para Motivar a Ingestão de Água. In Anais Estendidos do XVII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (Belém). SBC, Porto Alegre, RS, Brasil. DOI: 10.5753/ihc.2018.4204
José Cezar de Souza Filho, Marcos Randel Freitas Brito, and Andréia Libório Sampaio. 2020. Comparing heuristic evaluation and MALTU model in interaction evaluation of ubiquitous systems. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems. 1–10.
José Cezar de Souza Filho, Andréia Libório Sampaio, and Eduardo Luzeiro Feitosa. 2022. Hidden Things and Implicit Interaction: Looking Back on How Invisibility Evolved as a Quality-in-Use Attribute. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’22). Association for Computing Machinery, New York, NY, USA, Article 19, 11 pages. DOI: 10.1145/3554364.3559136
Carlos Eduardo Almeida Feitosa. 2022. Estendendo o modelo maltu para a avaliação da usabilidade e experiência do usuário em sistemas ubíquos. (2022).
Lavínia Matoso Freitas, Thiago Hellen O. da Silva, and Marília Soares Mendes. 2016. Evaluation of Spotify: An Evaluation Textual Experience Using the Maltu Methodology. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems (São Paulo, Brazil) (IHC ’16). Association for Computing Machinery, New York, NY, USA, Article 50, 4 pages. DOI: 10.1145/3033701.3033752
Hannu Karvonen and Tuomo Kujala. 2014. Designing and evaluating ubicomp characteristics of intelligent in-car systems. Advances in Ergonomics In Design, Usability & Special Populations: Part I 16 (2014), 27.
Hannu Korhonen, Juha Arrasvuori, and Kaisa Väänänen-Vainio-Mattila. 2010. Let users tell the story: evaluating user experience with experience reports. In CHI’10 Extended Abstracts on Human Factors in Computing Systems. 4051–4056.
Panos E Kourouthanassis, George M Giaglis, and Dimitrios C Karaiskos. 2008. Delineating the degree of’pervasiveness’ in pervasive information systems: an assessment framework and design implications. In 2008 Panhellenic Conference on Informatics. IEEE, 251–255.
Marília Soares Mendes. 2015. MALTU–Um modelo para avaliação da interação em sistemas sociais a partir da linguagem textual do usuário. (2015).
Marilia S Mendes and Elizabeth Furtado. 2018. An Experience of Textual Evaluation Using the MALTU Methodology. In Social Computing and Social Media. Technologies and Analytics: 10th International Conference, SCSM 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15-20, 2018, Proceedings, Part II 10. Springer, 236–246.
Vangelis Metsis, Zhengyi Le, Yu Lei, and Fillia Makedon. 2008. Towards an evaluation framework for assistive environments. In Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments. 1–8.
Jakob Nielsen and Rolf Molich. 1990. rochas. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Seattle, Washington, USA) (CHI ’90). Association for Computing Machinery, New York, NY, USA, 249–256. DOI: 10.1145/97243.97281
Stavroula Ntoa, George Margetis, Margherita Antona, and Constantine Stephanidis. 2021. User experience evaluation in intelligent environments: A comprehensive framework. Technologies 9, 2 (2021), 41.
Erica R Oliveira, Luiz CS Luz, and Raquel O Prates. 2008. Aplicação semi-estruturada do método de inspeção semiótica: estudo de caso para o domínio educacional. In Proceedings of the VIII Brazilian Symposium on Human Factors in Computing Systems. 50–59.
Thomas Olsson and Markus Salo. 2012. Narratives of satisfying and unsatisfying experiences of current mobile augmented reality applications. In Proceedings of the SIGCHI conference on human factors in computing systems. 2779–2788.
Daniela Quiñones and Cristian Rusu. 2017. How to develop usability heuristics: A systematic literature review. Computer Standards & Interfaces 53 (2017), 89–122. DOI: 10.1016/j.csi.2017.03.009
Larissa Castro Rocha. 2017. HUbis: heurísticas de usabilidade para avaliar sistemas ubíquos. (2017).
Cristian Rusu, Silvana Roncagliolo, Virginica Rusu, and César Collazos. 2011. A Methodology to establish usability heuristics.(2011).
Rainara Maia Santos. 2014. Características e medidas de software para avaliação da qualidade da interação humano-computador em sistemas ubíquos. (2014).
Fernanda Luiza Siewerdt, Rainara Maia Carvalho, and Rossana M. C. Andrade. 2016. Recommendations for Usability Testing in Ubiquitous Applications. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems (São Paulo, Brazil) (IHC ’16). Association for Computing Machinery, New York, NY, USA, Article 47, 4 pages. DOI: 10.1145/3033701.3033749
Alexandre N Tuch, Rune Trusell, and Kasper Hornbæk. 2013. Analyzing users’ narratives to understand experience with interactive products. In Proceedings of the SIGCHI conference on human factors in computing systems. 2079–2088.
Parvaneh Asghari, Amir Masoud Rahmani, and Hamid Haj Seyyed Javadi. 2019. Internet of Things applications: A systematic review. Computer Networks 148 (2019), 241–261.
Simone Barbosa and Bruno Silva. 2010. Interação humano-computador. Elsevier Brasil.
Marcos Randel Freitas Brito. 2018. Explorando o modelo MALTU na avaliação de usabilidade e experiência de uso em um dispositivo ubíquo. (2018).
Rainara Maia Carvalho. 2017. Dealing with Conflicts Between Non-functional Requirements of UbiComp and IoT Applications. In 2017 IEEE 25th International Requirements Engineering Conference (RE). 544–549. DOI: 10.1109/RE.2017.51
Rainara Maia Carvalho. 2019. Correlate & lead: process and catalog of non-functional requirements correlations in ubicomp and iot systems. (2019).
Rainara Maia Carvalho, Rossana Maria de Castro Andrade, and Káthia Marçal de Oliveira. 2018. AQUArIUM-A suite of software measures for HCI quality evaluation of ubiquitous mobile applications. Journal of Systems and Software 136 (2018), 101–136.
Rainara Maia Carvalho, Rossana Maria de Castro Andrade, Káthia Marçal de Oliveira, Ismayle de Sousa Santos, and Carla Ilane Moreira Bezerra. 2017. Quality characteristics and measures for human–computer interaction evaluation in ubiquitous systems. Software Quality Journal 25 (2017), 743–795.
Clarisse Sieckenius De Souza. 2005. The semiotic engineering of human-computer interaction. MIT press.
Luiz Gustavo de Souza, Simone Diniz Junqueira Barbosa, and Tayana Conte. 2015. Evaluating a MoLIC Extension for Collaborative Systems Design. In Proceedings of the 14th Brazilian Symposium on Human Factors in Computing Systems (Salvador, Brazil) (IHC ’15). Association for Computing Machinery, New York, NY, USA, Article 15, 10 pages. DOI: 10.1145/3148456.3148471
José Cezar de Souza Filho, Marcos Randel Freitas Brito, Antonio Raimundo Rocha Mendonça, Marcos Vitor Martins, and Andréia Libório Sampaio. 2018. Hidrate Spark: Avaliando um Sistema Ubíquo para Motivar a Ingestão de Água. In Anais Estendidos do XVII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (Belém). SBC, Porto Alegre, RS, Brasil. DOI: 10.5753/ihc.2018.4204
José Cezar de Souza Filho, Marcos Randel Freitas Brito, and Andréia Libório Sampaio. 2020. Comparing heuristic evaluation and MALTU model in interaction evaluation of ubiquitous systems. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems. 1–10.
José Cezar de Souza Filho, Andréia Libório Sampaio, and Eduardo Luzeiro Feitosa. 2022. Hidden Things and Implicit Interaction: Looking Back on How Invisibility Evolved as a Quality-in-Use Attribute. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’22). Association for Computing Machinery, New York, NY, USA, Article 19, 11 pages. DOI: 10.1145/3554364.3559136
Carlos Eduardo Almeida Feitosa. 2022. Estendendo o modelo maltu para a avaliação da usabilidade e experiência do usuário em sistemas ubíquos. (2022).
Lavínia Matoso Freitas, Thiago Hellen O. da Silva, and Marília Soares Mendes. 2016. Evaluation of Spotify: An Evaluation Textual Experience Using the Maltu Methodology. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems (São Paulo, Brazil) (IHC ’16). Association for Computing Machinery, New York, NY, USA, Article 50, 4 pages. DOI: 10.1145/3033701.3033752
Hannu Karvonen and Tuomo Kujala. 2014. Designing and evaluating ubicomp characteristics of intelligent in-car systems. Advances in Ergonomics In Design, Usability & Special Populations: Part I 16 (2014), 27.
Hannu Korhonen, Juha Arrasvuori, and Kaisa Väänänen-Vainio-Mattila. 2010. Let users tell the story: evaluating user experience with experience reports. In CHI’10 Extended Abstracts on Human Factors in Computing Systems. 4051–4056.
Panos E Kourouthanassis, George M Giaglis, and Dimitrios C Karaiskos. 2008. Delineating the degree of’pervasiveness’ in pervasive information systems: an assessment framework and design implications. In 2008 Panhellenic Conference on Informatics. IEEE, 251–255.
Marília Soares Mendes. 2015. MALTU–Um modelo para avaliação da interação em sistemas sociais a partir da linguagem textual do usuário. (2015).
Marilia S Mendes and Elizabeth Furtado. 2018. An Experience of Textual Evaluation Using the MALTU Methodology. In Social Computing and Social Media. Technologies and Analytics: 10th International Conference, SCSM 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15-20, 2018, Proceedings, Part II 10. Springer, 236–246.
Vangelis Metsis, Zhengyi Le, Yu Lei, and Fillia Makedon. 2008. Towards an evaluation framework for assistive environments. In Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments. 1–8.
Jakob Nielsen and Rolf Molich. 1990. rochas. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Seattle, Washington, USA) (CHI ’90). Association for Computing Machinery, New York, NY, USA, 249–256. DOI: 10.1145/97243.97281
Stavroula Ntoa, George Margetis, Margherita Antona, and Constantine Stephanidis. 2021. User experience evaluation in intelligent environments: A comprehensive framework. Technologies 9, 2 (2021), 41.
Erica R Oliveira, Luiz CS Luz, and Raquel O Prates. 2008. Aplicação semi-estruturada do método de inspeção semiótica: estudo de caso para o domínio educacional. In Proceedings of the VIII Brazilian Symposium on Human Factors in Computing Systems. 50–59.
Thomas Olsson and Markus Salo. 2012. Narratives of satisfying and unsatisfying experiences of current mobile augmented reality applications. In Proceedings of the SIGCHI conference on human factors in computing systems. 2779–2788.
Daniela Quiñones and Cristian Rusu. 2017. How to develop usability heuristics: A systematic literature review. Computer Standards & Interfaces 53 (2017), 89–122. DOI: 10.1016/j.csi.2017.03.009
Larissa Castro Rocha. 2017. HUbis: heurísticas de usabilidade para avaliar sistemas ubíquos. (2017).
Cristian Rusu, Silvana Roncagliolo, Virginica Rusu, and César Collazos. 2011. A Methodology to establish usability heuristics.(2011).
Rainara Maia Santos. 2014. Características e medidas de software para avaliação da qualidade da interação humano-computador em sistemas ubíquos. (2014).
Fernanda Luiza Siewerdt, Rainara Maia Carvalho, and Rossana M. C. Andrade. 2016. Recommendations for Usability Testing in Ubiquitous Applications. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems (São Paulo, Brazil) (IHC ’16). Association for Computing Machinery, New York, NY, USA, Article 47, 4 pages. DOI: 10.1145/3033701.3033749
Alexandre N Tuch, Rune Trusell, and Kasper Hornbæk. 2013. Analyzing users’ narratives to understand experience with interactive products. In Proceedings of the SIGCHI conference on human factors in computing systems. 2079–2088.
Publicado
16/10/2023
Como Citar
FEITOSA, Carlos Eduardo Almeida; MONTEIRO, Ingrid Teixeira.
Evaluating a MALTU Extension for UbiComp and IoT Systems. In: SIMPÓSIO BRASILEIRO SOBRE FATORES HUMANOS EM SISTEMAS COMPUTACIONAIS (IHC), 22. , 2023, Maceió/AL.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
