User Studies without End-users? Analysis of the Proxy-users Involvement in the Development of Digital Products
Abstract
Introduction: Users participation is crucial for the creation of digital products that address their real needs. In some of these cases, the only available option to ensure that some form of user perspective is considered during the product development is count on proxy-users. Objective: This paper examines how research studies involving proxy-users are conducted in the development of digital products. Methodology: A dual methodological approach was employed: firstly, a tertiary study was conducted from a previous systematic literature review with 164 studies, and secondly, a survey with 69 professionals involved in the development of digital products. Results: The results of the tertiary study indicate that while proxy-user research employs common HCI methods, aspects such as methodological rigor, justification for excluding end users, and proxy-users’ familiarity with the target audience are often unclear. The survey revealed diverse information sources on the users needs and contexts mentioned by respondents, and the reasons for not involving end users. To summarize the research findings, this study presents a set of 10 recommendations for industry professionals and researchers when involving proxy-users in their studies.
References
Almeida, C., Filho, P., Rosal, I., Gonçalves, R., Aguilar, P., Nunes, M., e Martins, M. (2024). Proposta de tecnologia assistiva para auxiliar pessoas com deficiência visual na detecção de obstáculos. In Anais do LI Seminário Integrado de Software e Hardware, pages 288–299, Porto Alegre, RS, Brasil. SBC.
Amati, C., Mitra, N. J., e Weyrich, T. (2014). A study of image colourfulness. In Proceedings of the Workshop on Computational Aesthetics, CAe ’14, page 23–31, New York, NY, USA. Association for Computing Machinery.
Badam, S. K., Chandrasegaran, S., Elmqvist, N., e Ramani, K. (2014). Tracing and sketching performance using blunt-tipped styli on direct-touch tablets. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, AVI ’14, page 193–200, New York, NY, USA. Association for Computing Machinery.
Badam, S. K., Zhao, J., Sen, S., Elmqvist, N., e Ebert, D. (2016). Timefork: Interactive prediction of time series. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, page 5409–5420, New York, NY, USA. Association for Computing Machinery.
Balata, J., Cmolik, L., e Mikovec, Z. (2014). On the selection of 2d objects using external labeling. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, page 2255–2258, New York, NY, USA. Association for Computing Machinery.
Balata, J., Mikovec, Z., e Neoproud, L. (2015). Blindcamera: Central and golden-ratio composition for blind photographers. In Proceedings of the Mulitimedia, Interaction, Design and Innnovation, MIDI ’15, New York, NY, USA. Association for Computing Machinery.
Barbosa, S. D. J., Silva, B. d., Silveira, M. S., Gasparini, I., Darin, T., e Barbosa, G. D. J. (2021). Interação humano-computador e experiência do usuario.
Benyon, D. (2011). Interação humano-computador. Person Prentice Hall.
Birmingham, C., Hu, Z., Mahajan, K., Reber, E., e Mataric,í M. J. (2020). Can i trust you? a user study of robot mediation of a support group. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 8019–8026.
Boyd-Graber, J. L., Nikolova, S. S., Moffatt, K. A., Kin, K. C., Lee, J. Y., Mackey, L. W., Tremaine, M. M., e Klawe, M. M. (2006). Participatory design with proxies: developing a desktop-pda system to support people with aphasia. In Proceedings of the SIGCHI conference on Human Factors in computing systems, pages 151–160.
Brazier, D. e Harvey, M. (2017). Strangers in a strange land: A study of second language speakers searching for e-services. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval, CHIIR ’17, page 281–284, New York, NY, USA. Association for Computing Machinery.
Bull, E. M., van der Cruyssen, L., Vágó, S., Király, G., Arbour, T., e van Dijk, L. (2024). Designing for agricultural digital knowledge exchange: applying a user-centred design approach to understand the needs of users. The Journal of Agricultural Education and Extension, 30(1):43–68.
Church, K., Ferreira, D., Banovic, N., e Lyons, K. (2015). Understanding the challenges of mobile phone usage data. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ’15, page 504–514, New York, NY, USA. Association for Computing Machinery.
Coleti, T., Martins, V., Milani, M., Trindade, D., e Merlin, J. (2024a). Design centrado no usuário para proposta de uma tecnologia assistiva para coordenação motora fna. In Anais do V Workshop sobre as Implicações da Computação na Sociedade, pages 36–46, Porto Alegre, RS, Brasil. SBC.
Coleti, T., Tirabassi, R., Almeida, G., Oliveira, G., Souza, V., Nobrega, J., e Luis, G. (2024b). Design centrado no usuário na construção de um sistema de gestão de dados para a equoterapia. In Anais Estendidos do XXIII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, pages 51–55, Porto Alegre, RS, Brasil. SBC.
da Silva, M. M., Chaves, L. S., Júnior, C. A. F., Guerra, C. S. D., Fernandes, S. R. L., Aguilar, P. A. C., Monteiro, I. T., e Sampaio, A. L. (2021). Wearable device in the form of glasses to assist the visually impaired in detecting obstacles. In Proceedings of the XX Brazilian Symposium on Human Factors in Computing Systems, IHC ’21, New York, NY, USA. Association for Computing Machinery.
de Oliveira, C. L. T., de Almeida Silva, A. T., de Morais, J. M. a., e Mota, M. P. (2020). Chartvision: accessible vertical bar charts. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems, IHC ’20, New York, NY, USA. Association for Computing Machinery.
de Sousa Silva, P. V., de Souza, A. J., e Monteiro, I. (2024). Process and challenges in designing data registration via fle import in an administrative e-commerce system. In Proceedings of the 26th International Conference on Enterprise Information Systems - Volume 2: ICEIS, pages 390–401. INSTICC, SciTePress.
Eason, K. D. (1989). Information technology and organisational change. CRC Press.
Ferreira, B., Kalinowski, M., Gomes, M. V. d. C., Marques, M. C., Lopes, H., e D.J. Barbosa, S. (2021). Investigating problem definition and end-user involvement in agile projects that use lean inceptions. In Proceedings of the XX Brazilian Symposium on Software Quality, SBQS ’21, New York, NY, USA. Association for Computing Machinery.
Frison, A.-K., Wintersberger, P., Riener, A., e Schartmüller, C. (2017). Driving hotzenplotz: A hybrid interface for vehicle control aiming to maximize pleasure in highway driving. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’17, page 236–244, New York, NY, USA. Association for Computing Machinery.
Gomes, D. (2022). Pesquisas de design centrado no usuário. o dia a dia na rethink tecnologia, uma consultoria de produto digital. In Anais Estendidos do XXI Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, pages 24–31, Porto Alegre, RS, Brasil. SBC.
Gonçalves, F., Campos, P., Hanna, J., e Ashby, S. (2015). You’re the voice: Evaluating user interfaces for encouraging underserved youths to express themselves through creative writing. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition, C&C ’15, page 63–72, New York, NY, USA. Association for Computing Machinery.
Gossen, T., Höbel, J., e Nürnberger, A. (2014). A comparative study about children’s and adults’ perception of targeted web search engines. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, page 1821–1824, New York, NY, USA. Association for Computing Machinery.
Gould, J. D. e Lewis, C. (1985). Designing for usability: key principles and what designers think. Communications of the ACM, 28(3):300–311.
Henley, A. Z. e Fleming, S. D. (2014). The patchworks code editor: toward faster navigation with less code arranging and fewer navigation mistakes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, page 2511–2520, New York, NY, USA. Association for Computing Machinery.
Junior, A. S., Castro, M., Reinbold, I., Teles, J., Silva, B., e Darin, T. (2023). Orbis 2.0: Um toolkit digital de técnicas do design centrado no usuário. In Anais Estendidos do XXII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, pages 139–143, Porto Alegre, RS, Brasil. SBC.
Kairam, S. e Heer, J. (2016). Parting crowds: Characterizing divergent interpretations in crowdsourced annotation tasks. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, CSCW ’16, page 1637–1648, New York, NY, USA. Association for Computing Machinery.
Keele, S. et al. (2007). Guidelines for performing systematic literature reviews in software engineering.
Kelly, D., Arguello, J., Edwards, A., e Wu, W.-c. (2015). Development and evaluation of search tasks for iir experiments using a cognitive complexity framework. In Proceedings of the 2015 International Conference on The Theory of Information Retrieval, ICTIR ’15, page 101–110, New York, NY, USA. Association for Computing Machinery.
Kryvoruchca, G., Corrêa, L., Parizi, R., e Marczak, S. (2020). The use of design thinking in a global information technology company. In Anais da IV Escola Regional de Engenharia de Software, pages 154–163, Porto Alegre, RS, Brasil. SBC.
Lazar, J., Feng, J. H., e Hochheiser, H. (2017). Research methods in human-computer interaction. Morgan Kaufmann.
Lievesley, M. A. e Yee, J. S. (2007). Surrogate users: a pragmatic approach to defning user needs. In CHI’07 extended abstracts on Human factors in computing systems, pages 1789–1794.
Lobão, L., Rabaioli, B., Gauthier, E., Araújo, L., e Meireles, M. (2023). The use of design thinking in creating projects and improving customer value perception: an experience report on software projects at a research and development institute in amazonas. In Anais do XXII Simpósio Brasileiro de Qualidade de Software, page 264–273, Porto Alegre, RS, Brasil. SBC.
Matin, S., Asma, E., Allen, E., Mangwiro, L., Mjumira, R., Valle, M., Acemyan, C., Oden, M., Kawaza, K., Kortum, P., et al. (2023). Proxy users for usability testing of medical devices for use in sub-saharan africa. In Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, volume 12, pages 208–212. SAGE Publications Sage CA: Los Angeles, CA.
Melenhorst, M., Novak, J., Micheel, I., Larson, M., e Boeckle, M. (2015). Bridging the utilitarian-hedonic divide in crowdsourcing applications. In Proceedings of the Fourth International Workshop on Crowdsourcing for Multimedia, CrowdMM ’15, page 9–14, New York, NY, USA. Association for Computing Machinery.
Moreno de Oliveira, G., Nascimento Carvalho, A., Lamego, B. S., Monteiro, I. T., Gonçalves, E. J. T., e Basilio, A. F. (2022). Reporting the application of user experience tools and proxy users in an industrial process based on double diamond. In International Conference on Human-Computer Interaction, pages 57–74. Springer.
Neris, V. P. A., Rosa, J. C. S., Maciel, C., Pereira, V. C., Galvão, V. F., e Arruda, I. L. (2024). Grandihc-br 2025-2035 - gc4: Sociocultural aspects in human-computer interaction. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems, IHC ’24, New York, NY, USA. Association for Computing Machinery.
Nielsen, J. (1994). Usability engineering. Morgan Kaufmann.
Norman, D. e Deiró, A. (2006). O design do dia a dia. Rocco.
Norman, D. A. e Draper, S. W. (1986). User centered system design: New perspectives on human-computer interaction. 1986. Hillsdale, NJ.
Novak, G. e Lundberg, L. (2015). Usability evaluation of an m-commerce system using proxy users. In HCI International 2015-Posters’ Extended Abstracts: International Conference, HCI International 2015, Los Angeles, CA, USA, August 2-7, 2015. Proceedings, Part II 17, pages 164–169. Springer.
Oliveira, G., Nascimento Carvalho, A., Lamego, B. S., Monteiro, I. T., Gonçalves, E. J. T., e Basilio, A. F. (2022). Reporting the application of user experience tools and proxy users in an industrial process based on double diamond. In Soares, M. M., Rosenzweig, E., e Marcus, A., editors, Design, User Experience, and Usability: UX Research, Design, and Assessment, pages 57–74, Cham. Springer International Publishing.
Ritter, E. F., Baxter, D. G., e Churchill, F. E. (2014). Foundations for designing user-centered systems: What system designers need to know about people. Springer.
Rogers, Y., Sharp, H., e Preece, J. (2013). Design de interação. Bookman Editora.
Salinas, E., Cueva, R., e Paz, F. (2020). A systematic review of user-centered design techniques. In Marcus, A. e Rosenzweig, E., editors, Design, User Experience, and Usability. Interaction Design, pages 253–267, Cham. Springer International Publishing.
Salminen, J., Jung, S.-g., Kamel, A., Froneman, W., e Jansen, B. J. (2022). Who is in the sample? an analysis of real and surrogate users as participants in user study research in the information technology felds. PeerJ Computer Science, 8:e1136.
Savolainen, K. (2021). User-centred design without involving users: A longitudinal case study in a human-centred-design–mature company. The Design Journal, 24(6):887–905.
Silva, P., de Souza, A., e Teixeira, I. (2024). Experience report on innovation in e-commerce: Research and design of an assisted remote sales application. In Anais do XXIII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, pages 81–97, Porto Alegre, RS, Brasil. SBC.
Silva Junior, D. P. d., Alves, D. D., Carneiro, N., Matos, E. d. S., Baranauskas, M. C. C., e Mendoza, Y. L. M. (2024). Grandihc-br 2025-2035 - gc1: New theoretical and methodological approaches in hci. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems, IHC ’24, New York, NY, USA. Association for Computing Machinery.
Sjölinder, M., Scandurra, I., Avatare Nou, A., e Kolkowska, E. (2017). Using care professionals as proxies in the design process of welfare technology – perspectives from municipality care. In Zhou, J. e Salvendy, G., editors, Human Aspects of IT for the Aged Population. Aging, Design and User Experience, pages 184–198, Cham. Springer International Publishing.
Soares, J. M. S., Guerra, C. S. D., da Silva, M. M., Chaves, L. S., Sampaio, A. L., e Al-Alam, W. G. a. (2021). Smart cane: Laser guide as an inclusion tool for the visually impaired. In Proceedings of the XX Brazilian Symposium on Human Factors in Computing Systems, IHC ’21, New York, NY, USA. Association for Computing Machinery.
Sofan, N. M., Hashim, A. S., e Sarlan, A. (2021). Multimedia elements in designing mobile app interface for autistic children: proxy user perspective. In 2021 International Conference on Computer & Information Sciences (ICCOINS), pages 96–101. IEEE.
Tamine, L. e Soulier, L. (2015). Understanding the impact of the role factor in collaborative information retrieval. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, CIKM ’15, page 43–52, New York, NY, USA. Association for Computing Machinery.
Teixeira, B., Ferreira, B., Damasceno, A., Barbosa, S. D. J., Novello, C., Villamizar, H., Kalinowski, M., Silva, T., Chueke, J., Lopes, H., Kuramoto, A., Itagyba, B., Salgado, C., Comandulli, S., Fischer, M., e Fialho, L. (2021). Lessons learned from a lean r&d project. In Proceedings of the 23rd International Conference on Enterprise Information Systems - Volume 2: ICEIS, pages 345–352. INSTICC, SciTePress.
Wyatt, S. M. (2003). Non-users also matter: The construction of users and non-users of the internet. Now users matter: The co-construction of users and technology, pages 67–79.
