UX-Tips: A UX evaluation technique to support the identification of software application problems
Resumo
User Experience (UX) is a quality attribute that plays an important role in the quality of software applications, as it addresses users' feelings about a given product. Thus, evaluating the UX is essential to improve its quality. However, most UX evaluation techniques limit to indicate whether the UX was positive or negative, which makes it challenging to identify what must be corrected by the software development team. This paper presents UX-Tips (User eXperience for Interactive ProductS), a technique developed to deal with this limitation, whose main objective is to allow the identification of problems related to UX. We also present two empirical studies conducted to evaluate UX-Tips. In the first study, we assessed the feasibility of the technique concerning its primary goal. In the second study, we compared the performance of UX-Tips with a similar technique. The results showed that UX-Tips allowed the subjects to indicate whether the UX was negative, and the specific problems they detected. It also allowed more effective and efficient evaluations, in addition to being considered easier to use. In practice, UX-Tips enable the software development team to identify the problems that impair users' experience, which should be fixed in order to provide a good UX.
Palavras-chave:
Software Quality, Verification and Validation, User Experience, Software evaluation
Referências
J. A. Bargas-Avila and K. Hornbæk (2011). "Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience". In Proceedings of the SIGCHI conference on human factors in computing systems, pp. 2689--2698.
H. Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences, Lawrence Erlbaum Hillsdale, New Jersey.
E.L.-C. Law, V. Roto, M. Hassenzahl, A. P. Vermeeren, J. Kort (2009) Understanding, scoping and defining user experience: a survey approach. In: Proceedings of the SIGCHI conference on human factors in computing systems. pp. 719--728.
E. L. C. Law, and S. Abrahão (2014). Interplay between User Experience (UX) evaluation and system development. International Journal of Human-Computer Studies, 72(6), pp. 523--525.
A. Fernandez, S. Abrahão, E. Insfran, and M. Matera (2012). "Further analysis on the validation of a usability inspection method for model-driven web development". In Proceedings of International symposium on Empirical software engineering and measurement, Lund, Sweden, pp. 153--156.
A. Fernandez, S. Abrahão, E. Insfran, M. Matera (2013). Usability Inspection in Model-driven Web Development: Empirical Validation in WebML". In Proceedings of 16th International Conference on Model Driven Engineering Languages and Systems, pp. 740--756.
X. Ferre, N. Juristo and A. M. Moreno (2005). Framework for Integrating Usability Practices into the Software Process. In: Bomarius F., Komi-Sirviö S. (eds) Product Focused Software Process Improvement. PROFES 2005. Lecture Notes in Computer Science, vol 3547. Springer, Berlin, Heidelberg.
H. M. Hassan and G. H. Galal-Edeen, G. H. (2017). From usability to user experience. In 2017 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), pp. 216--222.
M. Hassenzahl, S. Diefenbach and A. Göritz (2010). Needs, affect, and interactive products -- Facets of user experience. Interacting with Computers. 22, 353--362.
M. Hassenzahl. 2018. The thing and I: understanding the relationship between user and product. Funology 2, Springer, Cham.
ISO 9241-210 (2010). Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems.
ISO/IEC 25010 (2011). Systems and software engineering - SquaRE - Software product Quality Requirements and Evaluation: System and Software Quality Models).
ISO 9241-11 (2017). Ergonomics of human-system interaction - Part 11: Usability: Definitions and concepts.
G. H. J. Johannessen and K. Hornbæk (2014). Must evaluation methods be about usability? Devising and assessing the utility inspection method. Behaviour & Information Technology, 33(2), pp. 195--206.
N. Aquino, J. Vanderdonckt, N. Condori-Fernández, Ó. Dieste, and Ó. Pastor. (2010). Usability evaluation of multi-device/platform user interfaces generated by model-driven engineering. In Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (p. 30). ACM.
N. Juristo, A. M. Moreno. 2001. Basics of Software Engineering Experimentation. Kluwer Academic Publishers.
N. Juristo, A. M. Moreno and M.-I. Sanchez-Segura. (2007). Guidelines for eliciting usability functionalities. IEEE Transactions on Software Engineering, 33(11), pp. 744--758.
V. Kampenes, T. Dybå, J. E. Hannay, and D. I. K. Sjøberg (2007). Systematic review: A systematic review of effect size in software engineering experiments. Inf. Softw. Technol. 49(11-12), pp. 1073--1086.
H. B. Mann, D. R. Whitney. (1947). On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. In Annals of Mathematical Statistics, 18(1), pp. 50--60.
J. Nielsen and R. Molich (1990). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 249--256.
D. Norman, J. Miller, and A. Henderson (1995). What you see, some of what's in the future, and how we go about doing it: HI at Apple Computer. In Conference companion on Human factors in computing systems, p. 155.
H. Petrie, and N. Bevan. (2009). The Evaluation of Accessibility, Usability, and User Experience. The universal access handbook, CRC Press, Hoboken, NJ, USA.
I. Pettersson, F. Lachner, A. K. Frison, A. Riener, and A. Butz (2018). A Bermuda Triangle? In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 461.
E. G. Pinheiro, L. A. Lopes, T. U. Conte, L. A. M. Zaina (2018). The contribution of non-technical stakeholders on the specification of UX requirements: an experimental study using the proto-persona technique. In Proceedings of the Brazilian Symposium on Software Engineering, pp. 92--101.
S. Rajeshkumar, R. Omar and M. Mahmud (2013). Taxonomies of user experience (UX) evaluation methods. In 2013 International Conference on Research and Innovation in Information Systems (ICRIIS), pp. 533--538.
L. C. Marques, P. G. F. Matsubara, W. T. Nakamura, I. S. Wiese, L. A. M. Zaina, T. U. Conte. (2019). Relatório Técnico. Disponível em: https://doi.org/10.6084/m9.figshare.8111234
L. Rivero, and T. Conte (2017). A systematic mapping study on research contributions on UX evaluation technologies". In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems.
J. B. da Silva Júnior, and A. H. Kronbauer (2018). A Study of Hedonic Experience Related to UX Capture Techniques. In Proceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems p. 4.
J. I. G. Urrutia, E. Brangier and L. Cessat (2017). "Is a Holistic Criteria-Based Approach Possible in User Experience?". In International Conference of Design, User Experience, and Usability, pp. 395--409.
N. M. C. Valentim, J. Rabelo, A. C. Oran, T. Conte, T. and S. Marczak (2015). "A controlled experiment with usability inspection techniques applied to use case specifications: comparing the MIT 1 and the UCE techniques". In 2015 ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 206--215.
A. P. Vermeeren, L-C. E. Law, V. Roto, M. Obrist, J. Hoonhout, and K. Väänänen-Vainio-Mattila (2010). User experience evaluation methods: current state and development needs. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, pp. 521--530.
C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, B. and A. Wesslén (2012). Experimentation in software engineering. Springer Science & Business Media.
H. Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences, Lawrence Erlbaum Hillsdale, New Jersey.
E.L.-C. Law, V. Roto, M. Hassenzahl, A. P. Vermeeren, J. Kort (2009) Understanding, scoping and defining user experience: a survey approach. In: Proceedings of the SIGCHI conference on human factors in computing systems. pp. 719--728.
E. L. C. Law, and S. Abrahão (2014). Interplay between User Experience (UX) evaluation and system development. International Journal of Human-Computer Studies, 72(6), pp. 523--525.
A. Fernandez, S. Abrahão, E. Insfran, and M. Matera (2012). "Further analysis on the validation of a usability inspection method for model-driven web development". In Proceedings of International symposium on Empirical software engineering and measurement, Lund, Sweden, pp. 153--156.
A. Fernandez, S. Abrahão, E. Insfran, M. Matera (2013). Usability Inspection in Model-driven Web Development: Empirical Validation in WebML". In Proceedings of 16th International Conference on Model Driven Engineering Languages and Systems, pp. 740--756.
X. Ferre, N. Juristo and A. M. Moreno (2005). Framework for Integrating Usability Practices into the Software Process. In: Bomarius F., Komi-Sirviö S. (eds) Product Focused Software Process Improvement. PROFES 2005. Lecture Notes in Computer Science, vol 3547. Springer, Berlin, Heidelberg.
H. M. Hassan and G. H. Galal-Edeen, G. H. (2017). From usability to user experience. In 2017 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), pp. 216--222.
M. Hassenzahl, S. Diefenbach and A. Göritz (2010). Needs, affect, and interactive products -- Facets of user experience. Interacting with Computers. 22, 353--362.
M. Hassenzahl. 2018. The thing and I: understanding the relationship between user and product. Funology 2, Springer, Cham.
ISO 9241-210 (2010). Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems.
ISO/IEC 25010 (2011). Systems and software engineering - SquaRE - Software product Quality Requirements and Evaluation: System and Software Quality Models).
ISO 9241-11 (2017). Ergonomics of human-system interaction - Part 11: Usability: Definitions and concepts.
G. H. J. Johannessen and K. Hornbæk (2014). Must evaluation methods be about usability? Devising and assessing the utility inspection method. Behaviour & Information Technology, 33(2), pp. 195--206.
N. Aquino, J. Vanderdonckt, N. Condori-Fernández, Ó. Dieste, and Ó. Pastor. (2010). Usability evaluation of multi-device/platform user interfaces generated by model-driven engineering. In Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (p. 30). ACM.
N. Juristo, A. M. Moreno. 2001. Basics of Software Engineering Experimentation. Kluwer Academic Publishers.
N. Juristo, A. M. Moreno and M.-I. Sanchez-Segura. (2007). Guidelines for eliciting usability functionalities. IEEE Transactions on Software Engineering, 33(11), pp. 744--758.
V. Kampenes, T. Dybå, J. E. Hannay, and D. I. K. Sjøberg (2007). Systematic review: A systematic review of effect size in software engineering experiments. Inf. Softw. Technol. 49(11-12), pp. 1073--1086.
H. B. Mann, D. R. Whitney. (1947). On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. In Annals of Mathematical Statistics, 18(1), pp. 50--60.
J. Nielsen and R. Molich (1990). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 249--256.
D. Norman, J. Miller, and A. Henderson (1995). What you see, some of what's in the future, and how we go about doing it: HI at Apple Computer. In Conference companion on Human factors in computing systems, p. 155.
H. Petrie, and N. Bevan. (2009). The Evaluation of Accessibility, Usability, and User Experience. The universal access handbook, CRC Press, Hoboken, NJ, USA.
I. Pettersson, F. Lachner, A. K. Frison, A. Riener, and A. Butz (2018). A Bermuda Triangle? In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 461.
E. G. Pinheiro, L. A. Lopes, T. U. Conte, L. A. M. Zaina (2018). The contribution of non-technical stakeholders on the specification of UX requirements: an experimental study using the proto-persona technique. In Proceedings of the Brazilian Symposium on Software Engineering, pp. 92--101.
S. Rajeshkumar, R. Omar and M. Mahmud (2013). Taxonomies of user experience (UX) evaluation methods. In 2013 International Conference on Research and Innovation in Information Systems (ICRIIS), pp. 533--538.
L. C. Marques, P. G. F. Matsubara, W. T. Nakamura, I. S. Wiese, L. A. M. Zaina, T. U. Conte. (2019). Relatório Técnico. Disponível em: https://doi.org/10.6084/m9.figshare.8111234
L. Rivero, and T. Conte (2017). A systematic mapping study on research contributions on UX evaluation technologies". In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems.
J. B. da Silva Júnior, and A. H. Kronbauer (2018). A Study of Hedonic Experience Related to UX Capture Techniques. In Proceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems p. 4.
J. I. G. Urrutia, E. Brangier and L. Cessat (2017). "Is a Holistic Criteria-Based Approach Possible in User Experience?". In International Conference of Design, User Experience, and Usability, pp. 395--409.
N. M. C. Valentim, J. Rabelo, A. C. Oran, T. Conte, T. and S. Marczak (2015). "A controlled experiment with usability inspection techniques applied to use case specifications: comparing the MIT 1 and the UCE techniques". In 2015 ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 206--215.
A. P. Vermeeren, L-C. E. Law, V. Roto, M. Obrist, J. Hoonhout, and K. Väänänen-Vainio-Mattila (2010). User experience evaluation methods: current state and development needs. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, pp. 521--530.
C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, B. and A. Wesslén (2012). Experimentation in software engineering. Springer Science & Business Media.
Publicado
23/09/2019
Como Citar
MARQUES, Leonardo; MATSUBARA, Patrícia; NAKAMURA, Walter; WIESE, Igor; ZAINA, Luciana; CONTE, Tayana.
UX-Tips: A UX evaluation technique to support the identification of software application problems. In: SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE (SBES), 33. , 2019, Salvador.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
