Textual evaluation vs. User testing: a comparative analysis
Resumo
One of the ways to evaluate a system is from Textual Evaluation. This type of evaluation consider the textual user’s opinions to infer aspects of the interaction with the system. Although this method covers many texts and consi- ders spontaneous narratives of the users, it takes a lot of time and effort. Some authors have reported on the need to compare evaluations techniques in order to investigate their effectiveness in revealing issue or to supplement the results of a systems assessment. This study presents a comparative analysis between the textual evaluation and user testing. A case study was performed evaluating the usability and user experience of a health app. As a result, the techniques were analyzed based on aspects that involved describing the results, resources needed and description of problems and users.
Palavras-chave:
Usability, User Experience, Usage Related Posts
Referências
Bach, C. and Scapin, D. L. (2010). Comparing inspections and user testing for the evaluation of virtual environments. Int. J. Hum. Comput. Interaction, 26:786–824.
Barbosa, S. and Silva, B. (2010). Interação Humano-Computador. Elsevier Brasil.
Bargas-Avila, J. A. and Hornbæk, K. (2011). Old wine in new bottles or novel challen- ges: a critical analysis of empirical studies of user experience. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 2689–2698. ACM.
Bim, S. A., de Castro Salgado, L. C., and Leita ̃o, C. F. (2016). Evaluation by inspection: Comparing methods of practical, cognitive and semiotic basis. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems, IHC ’16, pages 9:1–9:10, New York, NY, USA. ACM.
da Silva, T. H. O., Freitas, L. M., and Mendes, M. S. (2017). Beyond traditional evaluations: User’s view in app stores. In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems, IHC 2017, pages 15:1–15:10, New York, NY, USA. ACM.
de Souza Filho, J. C., Brito, M. R. F., Mendonç̧a, A. R. R., Martins, M. V., and Sampaio, A. L. (2018). Hidrate spark: Avaliando um sistema ubíquo para motivar a ingestã̃o de Á́gua.In Anais Estendidos do XVII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, Porto Alegre, RS, Brasil. SBC.
Fetter, M. and Gross, T. (2014). Lilole—a framework for lifelong learning from sensor data streams for predictive user modelling. In International Conference on Human- Centred Software Engineering, pages 126–143. Springer.
Freitas, L. M., da Silva, T. H. O., and Mendes, M. S. (2016). Evaluation of spotify: An evaluation textual experience using the maltu methodology. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems, IHC ’16, pages 50:1–50:4, New York, NY, USA. ACM.
Gray, W. D. and Salzman, M. C. (1998). Damaged merchandise? a review of experiments that compare usability evaluation methods. Hum.-Comput. Interact., 13(3):203–261.
Hedegaard, S. and Simonsen, J. G. (2013). Extracting usability and user experience information from online user reviews. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13, pages 2089–2098, New York, NY, USA. ACM.
Hedegaard, S. and Simonsen, J. G. (2014). Mining until it hurts: Automatic extraction of usability issues from online reviews compared to traditional usability evaluation. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, NordiCHI ’14, pages 157–166, New York, NY, USA. ACM.
Korhonen,H., Arrasvuori,J., and Va ̈a ̈na ̈nen-Vainio-Mattila, K. (2010). Letuserstell the story: Evaluating user experience with experience reports. In CHI ’10 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’10, pages 4051–4056, New York, NY, USA. ACM.
Mendes, M. S. (2015). MALTU - Um modelo para avaliação da interação em sistemas sociais a partir da linguagem textual do usuário. PhD thesis, Universidade Federal do Ceará (UFC).
MyFitnessPal (2019). Myfitnesspal - contador de calorias @ONLINE. Acesso em 02 de maio de 2019.
Nielsen, J. (2012). How many test users in a usability study? @ONLINE. Acesso em 18 de abril de 2019.
Norman, D. (2004). Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books.
Rogers, Y., Sharp, H., and Preece, J. (2013). Design de Interação - 3ed. Bookman Editora.
Barbosa, S. and Silva, B. (2010). Interação Humano-Computador. Elsevier Brasil.
Bargas-Avila, J. A. and Hornbæk, K. (2011). Old wine in new bottles or novel challen- ges: a critical analysis of empirical studies of user experience. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 2689–2698. ACM.
Bim, S. A., de Castro Salgado, L. C., and Leita ̃o, C. F. (2016). Evaluation by inspection: Comparing methods of practical, cognitive and semiotic basis. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems, IHC ’16, pages 9:1–9:10, New York, NY, USA. ACM.
da Silva, T. H. O., Freitas, L. M., and Mendes, M. S. (2017). Beyond traditional evaluations: User’s view in app stores. In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems, IHC 2017, pages 15:1–15:10, New York, NY, USA. ACM.
de Souza Filho, J. C., Brito, M. R. F., Mendonç̧a, A. R. R., Martins, M. V., and Sampaio, A. L. (2018). Hidrate spark: Avaliando um sistema ubíquo para motivar a ingestã̃o de Á́gua.In Anais Estendidos do XVII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, Porto Alegre, RS, Brasil. SBC.
Fetter, M. and Gross, T. (2014). Lilole—a framework for lifelong learning from sensor data streams for predictive user modelling. In International Conference on Human- Centred Software Engineering, pages 126–143. Springer.
Freitas, L. M., da Silva, T. H. O., and Mendes, M. S. (2016). Evaluation of spotify: An evaluation textual experience using the maltu methodology. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems, IHC ’16, pages 50:1–50:4, New York, NY, USA. ACM.
Gray, W. D. and Salzman, M. C. (1998). Damaged merchandise? a review of experiments that compare usability evaluation methods. Hum.-Comput. Interact., 13(3):203–261.
Hedegaard, S. and Simonsen, J. G. (2013). Extracting usability and user experience information from online user reviews. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13, pages 2089–2098, New York, NY, USA. ACM.
Hedegaard, S. and Simonsen, J. G. (2014). Mining until it hurts: Automatic extraction of usability issues from online reviews compared to traditional usability evaluation. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, NordiCHI ’14, pages 157–166, New York, NY, USA. ACM.
Korhonen,H., Arrasvuori,J., and Va ̈a ̈na ̈nen-Vainio-Mattila, K. (2010). Letuserstell the story: Evaluating user experience with experience reports. In CHI ’10 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’10, pages 4051–4056, New York, NY, USA. ACM.
Mendes, M. S. (2015). MALTU - Um modelo para avaliação da interação em sistemas sociais a partir da linguagem textual do usuário. PhD thesis, Universidade Federal do Ceará (UFC).
MyFitnessPal (2019). Myfitnesspal - contador de calorias @ONLINE. Acesso em 02 de maio de 2019.
Nielsen, J. (2012). How many test users in a usability study? @ONLINE. Acesso em 18 de abril de 2019.
Norman, D. (2004). Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books.
Rogers, Y., Sharp, H., and Preece, J. (2013). Design de Interação - 3ed. Bookman Editora.
Publicado
21/10/2019
Como Citar
HELLEN O. DA SILVA , Thiago; MATOSO FREITAS , Lavínia; SOARES MENDES , Marília; SUCUPIRA FURTADO , Elizabeth.
Textual evaluation vs. User testing: a comparative analysis. In: WORKSHOP SOBRE ASPECTOS DA INTERAÇÃO HUMANO-COMPUTADOR NA WEB SOCIAL (WAIHCWS), 10. , 2019, Vitória.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
p. 21-30.
ISSN 2596-0296.
DOI: https://doi.org/10.5753/waihcws.2019.7673.
