Uma Taxonomia de Abordagens Baseadas em LLMs para Automação da Inspeção de Usabilidade

• Ana Beatriz de Araújo Farias IFPB https://orcid.org/0009-0009-0491-1315
• Emanuel Dantas Filho IFPB https://orcid.org/0000-0002-8204-8731
• Danyllo Wagner Albuquerque IFPB https://orcid.org/0000-0001-5515-7812
• Bruno Neiva Moreno IFPB https://orcid.org/0000-0002-7166-2090

DOI: https://doi.org/10.5753/semish.2026.21830

Resumo

Este artigo propõe uma taxonomia de abordagens baseadas em Modelos de Linguagem de Grande Escala (LLMs) para automação da inspeção de usabilidade. Para isso, foi realizado um mapeamento sistemático da literatura em bases de dados científicas consolidadas, considerando publicações entre 2022 e 2025. A busca resultou em 401 estudos, dos quais 14 foram selecionados com base em critérios de exclusão e de qualidade. A taxonomia foi derivada de uma síntese qualitativa orientada pelas questões de pesquisa. Os resultados indicam a predominância de abordagens automatizadas, a adoção recorrente de modelos da família GPT-4 e o papel central do prompt engineering na condução das inspeções. Com base nesses padrões, a taxonomia organiza a literatura em cinco dimensões: estratégias de uso de LLMs, nível de automação, artefatos de entrada, referenciais de inspeção e formas de validação. A análise também evidenciou desafios recorrentes, como inconsistência nas respostas, falsos positivos, dificuldades de interpretação contextual, dependência de validação humana e concentração em modelos proprietários. Conclui-se que a taxonomia proposta não apenas organiza o estado da arte, mas também explicita lacunas e orienta uma agenda de pesquisa voltada ao desenvolvimento de abordagens mais robustas, comparáveis e confiáveis.

Referências

Bisante, A. et al. (2024). Enhancing interface design with ai: An exploratory study on a chatgpt-4-based tool for cognitive walkthrough inspired evaluations. In Proceedings of the 2024 International Conference on Advanced Visual Interfaces, AVI ’24, New York, NY, USA. Association for Computing Machinery.

Braun, V. and Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2):77–101.

Doğan, S., Betin-Can, A., and Garousi, V. (2014). Web application testing: A systematic literature review. Journal of Systems and Software, 91:174–201.

Duan, P., Cheng, C.-Y., Li, G., Hartmann, B., and Li, Y. (2024a). Uicrit: Enhancing automated design evaluation with a ui critique dataset. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, UIST ’24, New York, NY, USA. Association for Computing Machinery.

Duan, P., Warner, J., and Hartmann, B. (2023). Towards generating ui design feedback with llms. In Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23 Adjunct, New York, NY, USA. Association for Computing Machinery.

Duan, P., Warner, J., Li, Y., and Hartmann, B. (2024b). Generating automatic feedback on ui mockups with large language models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24, New York, NY, USA. Association for Computing Machinery.

Fernandez, A., Insfran, E., and Abrahão, S. (2011). Usability evaluation methods for the web: A systematic mapping study. Information and Software Technology, 53(8):789–817.

Guerino, G., Rodrigues, L., Capeleti, B., Mello, R. F., Freire, A., and Zaina, L. (2025). Can gpt-4o evaluate usability like human experts? a comparative study on issue identification in heuristic evaluation. Accepted at INTERACT 2025.

Hsueh, N.-L., Lin, H.-J., and Lai, L.-C. (2024). Applying large language model to user experience testing. Electronics, 13(23).

Huang, X., Zhang, H., and Ali Babar, M. (2018). Synthesizing qualitative research in software engineering: A critical review. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results, pages 41–44.

International Organization for Standardization (2018). Iso 9241-11:2018 — ergonomics of human-system interaction – part 11: Usability: Definitions and concepts. Disponível em: [link]. Acesso em: 10 maio 2025.

Kuric, E., Demcak, P., Krajcovic, M., and Lang, J. (2025). Systematic literature review of automation and artificial intelligence in usability issue detection.

Liu, J. (2025). Ai-powered automated and remote ux evaluation methods: A systematic literature review. In Proceedings of the 43rd ACM International Conference on Design of Communication, SIGDOC ’25, page 10–16, New York, NY, USA. Association for Computing Machinery.

Lu, Y., Yao, B., Gu, H., Huang, J., Wang, J., Li, Y., Gesi, J., He, Q., Li, T. J.-J., and Wang, D. (2025). Uxagent: An llm agent-based usability testing framework for web design. Also available as arXiv:2502.12561 and listed by Amazon Science as CHI 2025.

Lubos, S., Felfernig, A., Garber, D., Le, V. M., and Tran, T. N. T. (2025). Towards llm-based usability analysis for recommender user interfaces. CEUR Workshop Proceedings, 4027. Presented at the 12th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems (IntRS 2025).

Nickerson, R. C., Varshney, U., and Muntermann, J. (2013). A method for taxonomy development and its application in information systems. European Journal of Information Systems, 22(3):336–359.

Nielsen, J. and Mack, R. L. (1994). Usability Inspection Methods. John Wiley & Sons, New York, NY, USA.

Norman, D. A. (2006). O design do dia-a-dia. Rocco, Rio de Janeiro.

Petersen, K., Feldt, R., Mujtaba, S., and Mattsson, M. (2008). Systematic mapping studies in software engineering. In Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE), pages 68–77.

Petersen, K., Vakkalanka, S., and Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64:1–18.

Pourasad, A. E. and Maalej, W. (2025). Does genai make usability testing obsolete? Santos, S., Alves, R., Oliveira, F., and Araujo, F. (2025). Investigating llm-based tools to support usability, accessibility, user experience in hci activities: A systematic literature mapping. In Proceedings of the 31st Brazilian Symposium on Multimedia and the Web, pages 631–645, Porto Alegre, RS, Brasil. SBC.

Shin, S., Oh, J., and Lee, S. (2025). Can llms see what i see? a study on five prompt engineering techniques for evaluating ux on a shopping site. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA ’25, New York, NY, USA. Association for Computing Machinery.

Wu, J., Peng, Y.-H., Li, A., Swearngin, A., Bigham, J. P., and Nichols, J. (2024). Uiclip: A data-driven model for assessing user interface design.

Xiang, W., Zhu, H., Lou, S., Chen, X., Pan, Z., Jin, Y., Chen, S., and Sun, L. (2024). Simuser: Generating usability feedback by simulating various users interacting with mobile applications. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24, New York, NY, USA. Association for Computing Machinery.

Zhong, M., Chen, R., Chen, X., Fogarty, J., and Wobbrock, J. O. (2025). Screenaudit: Detecting screen reader accessibility errors in mobile apps using large language models. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI ’25, New York, NY, USA. Association for Computing Machinery.