Investigating LLM-based tools to support Usability, Accessibility, User Experience in HCI activities: A Systematic Literature Mapping
Resumo
This study presents a Systematic Literature Mapping (SLM) of recent research on the application of Large Language Models (LLMs) within Human–Computer Interaction (HCI). Drawing from 104 peer-reviewed articles published between 2021 and 2025 in the ACM Digital Library and IEEE Xplore, the mapping investigates the role of LLMs in supporting usability, accessibility, and user experience (UX) activities. LLMs, rooted in Natural Language Processing (NLP), have demonstrated widespread utility across various domains, including healthcare, education, and software engineering. Within HCI, their applications include conversational agents, accessibility enhancements, user journey modeling, prototyping tools, and adaptive user interfaces. The findings highlight how UX professionals have integrated LLM-based solutions into design workflows, leveraging these technologies for improved engagement, personalization, and collaboration. This research addresses 4 guiding questions concerning the domains of application, methodologies, and practitioner adoption of LLMs. The study contributes by (i) offering a comprehensive mapping of HCI-related LLM literature, (ii) identifying areas of active research, and (iii) uncovering emerging trends and open challenges that may inform future investigations.
Palavras-chave:
Generative AI, usability, accessibility, UX, HCI activities, systematic mapping
Referências
Wajdi Aljedaani, Marcelo Medeiros Eler, and PD Parthasarathy. 2025. Enhancing accessibility in software engineering projects with large language models (llms). In Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1. 25–31.
Wajdi Aljedaani, Abdulrahman Habib, Ahmed Aljohani, Marcelo Eler, and Yunhe Feng. 2024. Does ChatGPT Generate Accessible Code? Investigating Accessibility Challenges in LLM-Generated Source Code. In Proceedings of the 21st International Web for All Conference (Singapore, Singapore) (W4A ’24). Association for Computing Machinery, New York, NY, USA, 165–176. DOI: 10.1145/3677846.3677854
Rubén Alonso, Danilo Dessí, Antonello Meloni, Marco Murgia, Diego Reforgiato Recupero, and Giuseppe Scarpi. 2024. A Seamless ChatGPT Knowledge Plug-in for the Labour Market. IEEE Access 12 (2024), 155821–155837. DOI: 10.1109/ACCESS.2024.3485111
Riku Arakawa, Jill Fain Lehman, and Mayank Goel. 2024. Prism-q&a: Step-aware voice assistant on a smartwatch enabled by multimodal procedure tracking and large language models. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 4 (2024), 1–26.
Aram Bahrini, Mohammadsadra Khamoshifar, Hossein Abbasimehr, Robert J. Riggs, Maryam Esmaeili, Rastin Mastali Majdabadkohne, and Morteza Pasehvar. 2023. ChatGPT: Applications, Opportunities, and Threats. In 2023 SIEDS. 274– 279. DOI: 10.1109/SIEDS58326.2023.10137850
Jose Barambones, Cristian Moral, Angélica de Antonio, Ricardo Imbert, Loïc Martínez-Normand, and Elena Villalba-Mora. 2024. ChatGPT for Learning HCI Techniques: A Case Study on Interviews for Personas. IEEE Transactions on Learning Technologies 17 (2024), 1460–1475. DOI: 10.1109/TLT.2024..3386095
Simone Barbosa and Bruno Silva. 2010. Interação humano-computador. Elsevier Brasil.
Majid Behravan and Denis Gračanin. 2025. From Voices to Worlds: Developing an AI-Powered Framework for 3D Object Generation in Augmented Reality. In 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 150–155.
Benyamin Beyzaei, Saghar Talebipour, Ghazal Rafiei, Nenad Medvidović, and Sam Malek. 2025. Automated Test Transfer across Android Apps using Large Language Models. Proc. ACM Softw. Eng. 2, ISSTA, Article ISSTA098 (June 2025), 24 pages.
Anastasiia Birillo, Elizaveta Artser, Anna Potriasaeva, Ilya Vlasov, Katsiaryna Dzialets, Yaroslav Golubev, Igor Gerasimov, Hieke Keuning, and Timofey Bryksin. 2024. One step at a time: Combining llms and static analysis to generate next-step hints for programming tasks. In Proceedings of the 24th Koli Calling International Conference on Computing Education Research. 1–12.
Mariana Gomes Borges, Claiton Marques Correa, and Milene Selbach Silveira. 2024. Human+ Human or Human+ AI: A Comparative Analysis of a Narrative Visualization Design Guide. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems. 1–13.
Paul Brie, Nicolas Burny, Arthur Sluÿters, and Jean Vanderdonckt. 2023. Evaluating a Large Language Model on Searching for GUI Layouts. Proc. ACM Hum.-Comput. Interact. 7, EICS, Article 178 (June 2023), 37 pages. DOI: 10.1145/3593230
Yining Cao, Peiling Jiang, and Haijun Xia. 2025. Generative and Malleable User Interfaces with Generative and Evolving Task-Driven Data Model. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20.
Inês Carmo, Patrícia Costa, and Pedro Santana. 2024. Boosting children’s reading motivation with LLM-generated story crossovers. In 2024 International Conference on Graphics and Interaction (ICGI). IEEE, 1–8.
Chaoran Chen, Weijun Li, Wenxin Song, Yanfang Ye, Yaxing Yao, and Toby Jia-Jun Li. 2024. An empathy-based sandbox approach to bridge the privacy gap among attitudes, goals, knowledge, and behaviors. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–28.
Chen Chen, Cuong Nguyen, Thibault Groueix, Vladimir G Kim, and Nadir Weibel. 2024. Memovis: A GenAI-powered tool for creating companion reference images for 3d design feedback. ACM Transactions on Computer-Human Interaction 31, 5 (2024), 1–41.
Liuqing Chen, Zhaojun Jiang, Duowei Xia, Zebin Cai, Lingyun Sun, Peter Childs, and Haoyu Zuo. 2024. BIDTrainer: An LLMs-driven Education Tool for Enhancing the Understanding and Reasoning in Bio-inspired Design. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 676, 20 pages. DOI: 10.1145/3613904.3642887
Pei Chen, Jiayi Yao, Zhuoyi Cheng, Yichen Cai, Jiayang Li, Weitao You, and Lingyun Sun. 2025. CoExploreDS: Framing and Advancing Collaborative Design Space Exploration Between Human and AI. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 22, 20 pages. DOI: .org/10.1145/3706598.3713869
Sharfuddin Khan Chisty, Anika Tahsin Miami, and Jannatun Noor. 2025. Smart Sheba: Enhancing Elderly User Experience with LLM-Enabled Chatbots and User-Centered Design. In Proceedings of the 13th International Conference on Information & Communication Technologies and Development (Nairobi, Kenya) (ICTD ’24). Association for Computing Machinery, New York, NY, USA, 69–83. DOI: 10.1145/3700794.3700803
Sravan Kumar Chittimalla and Leela Krishna M Potluri. 2025. Enhancing Yoga Pose Recognition Using Large Language Models with Real-World Dataset Integration for Personalized Fitness Assistance. In 2024 International Conference on IT Innovation and Knowledge Discovery (ITIKD). 1–6. DOI: 10.1109/ITIKD63574.2025.11004828
Nicholas Davis, Jacob Sherson, and Janet Rafner. 2025. The Co-Creative Design Framework for Hybrid Intelligence. In Proceedings of the 2025 Conference on Creativity and Cognition (C&C ’25). Association for Computing Machinery, New York, NY, USA, 560–572. DOI: 10.1145/3698061.3726934
Gabriel Dengler, Peter Bazan, Reinhard German, Pooia Lalbakhsh, and Ariel Liebmann. 2024. A Conversational Human-Computer Interface for Smart Energy System Simulation Environments. In Proceedings of the Winter Simulation Conference (San Antonio, Texas, USA) (WSC ’23). IEEE Press, 2978–2989.
Ati Suci Dian Martha, Sri Widowati, Desy Puspa Rahayu, Muhammad Ilham Nursyawal, and Rayhan Rizqial Hariz. 2024. Examining Usability and Student Experience of ChatGPT as a Learning Tool on Moodle in Higher Education. In 2024 International Conference on Information Technology Systems and Innovation (ICITSI). 8–13. DOI: 10.1109/ICITSI65188.2024.10929431
Hyo Jin Do, Michelle Brachman, Casey Dugan, James M Johnson, Julia Lauer, Priyanshu Rai, and Qian Pan. 2024. Grounding with Structure: Exploring Design Variations of Grounded Human-AI Collaboration in a Natural Language Interface. Proceedings of the ACM on Human-Computer Interaction 8, CSCW2 (2024), 1–27.
Peitong Duan, JeremyWarner, Yang Li, and Bjoern Hartmann. 2024. Generating Automatic Feedback on UI Mockups with Large Language Models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 6, 20 pages. DOI: 10.1145/3613904.3642782
Fernanda Espinoza, Darren Cook, Chris R Butler, and Rafael A Calvo. 2023. Supporting dementia caregivers in Peru through chatbots: generative AI vs structured conversations. In Electronic Workshops in Computing. BCS Learning & Development, 89–98. DOI: 10.14236/ewic/BCSHCI2023.11
K. J. Kevin Feng, Q. Vera Liao, Ziang Xiao, Jennifer Wortman Vaughan, Amy X. Zhang, and David W. McDonald. 2025. Canvil: Designerly Adaptation for LLMPowered User Experiences. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, Article 932, 22 pages.
Victoria Firsanova. 2023. Towards Building a Mobile App for People on the Spectrum. In Companion Proceedings of the ACM Web Conference 2023 (Austin, TX, USA) (WWW ’23 Companion). Association for Computing Machinery, New York, NY, USA, 555–559. DOI: 10.1145/3543873.3587533
Claudia Flores-Saviaga, Benjamin V. Hanrahan, Kashif Imteyaz, Steven Clarke, and Saiph Savage*. 2025. The Impact of Generative AI Coding Assistants on Developers Who Are Visually Impaired. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 1164, 17 pages. DOI: 10.1145/3706598.3714008
Toru Furusawa. 2025. In-Vehicle Artificial Intelligence Agent System with Local Large Language Models for Agent Selection and Vehicle Control. In 2025 International Conference on Computing, Networking and Communications (ICNC). IEEE, 259–264.
Muhammet Kürşat Görmez, Murat Yılmaz, and Paul M Clarke. 2024. Large language models for software engineering: A systematic mapping study. In European Conference on Software Process Improvement, Vol. 2179. Springer, Cham, 64–79. DOI: 10.1007/978-3-031-71139-8_5
Joshua Gorniak, Yoon Kim, Donglai Wei, and Nam Wook Kim. 2024. VizAbility: Enhancing Chart Accessibility with LLM-based Conversational Interaction. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (Pittsburgh, PA, USA) (UIST ’24). Article 89, 19 pages.
Yi Guo, Danqing Shi, Mingjuan Guo, YanqiuWu, Nan Cao, and Qing Chen. 2024. Talk2Data: A Natural Language Interface for Exploratory Visual Analysis via Question Decomposition. ACM Trans. Interact. Intell. Syst. 14, 2, Article 8 (April 2024), 24 pages. DOI: 10.1145/3643894
Jihyeong Hong, Yokyung Lee, Dae Hyun Kim, DaEun Choi, Yeo-Jin Yoon, Gyucheol Lee, Zucheul Lee, and Juho Kim. 2024. A context-aware onboarding agent for metaverse powered by large language models. In Proceedings of the 2024 ACM Designing Interactive Systems Conference. 1857–1874.
Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large Language Models for Software Engineering: A Systematic Literature Review. ACM Trans. Softw. Eng. Methodol. 33, 8, Article 220 (Dec. 2024), 79 pages. DOI: 10.1145/3695988
Forrest Huang, Gang Li, Tao Li, and Yang Li. 2024. Automatic Macro Mining from Interaction Traces at Scale. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 1038, 16 pages. DOI: 10.1145/3613904.3642074
Faria Huq, Abdus Samee, David Chuan-En Lin, Alice Xiaodi Tang, and Jeffrey P Bigham. 2025. NoTeeline: Supporting Real-Time, Personalized Notetaking with LLM-Enhanced Micronotes. In Proceedings of the 30th International Conference on Intelligent User Interfaces. 1064–1081.
Srujana Inturi, Swarna Latha Potluri, Guda Vanitha, and B Indira. 2025. VirtualMe: A Large Language Model-Powered Chatbot for Enhanced Human Computer Interaction. In 2025 International Conference on Machine Learning and Autonomous Systems (ICMLAS). IEEE, 626–634.
Dhiraj Jadhav, Sakshee Agrawal, Sakshi Jagdale, Pranav Salunkhe, and Raj Salunkhe. 2024. AI-Driven Text-to-Multimedia Content Generation: Enhancing Modern Content Creation. In 2024 8th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC). 1610–1615. DOI: 10.1109/ISMAC61858.2024.10714771
Jazette Johnson, Hira Jamshed, Rachael Zuppke, Amanda Leggett, Emily Mower Provost, and Robin N. Brewer. 2025. “I felt listened to”: Evaluating an AI-Powered Reflection Tool for Care Partners. ACM Trans. Access. Comput. (April 2025). Just Accepted.
Aayush Kapoor and Sujala D. Shetty. 2024. Enhancing Healthcare Information Accessibility Through a Generative Medical Chatbot. In 2024 ICETCS. 1–4.
Kai Karren, Michael Schmitz, and Stefan Schaffer. 2024. Improving conversational user interfaces for citizen complaint management through enhanced contextual feedback. In Proceedings of the 6th ACM Conference on Conversational User Interfaces. 1–11.
Yewon Kim, Sung-Ju Lee, and Chris Donahue. 2025. Amuse: Human-AI Collaborative Songwriting with Multimodal Inspirations. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–28.
Barbara Kitchenham, Stuart Charters, et al. 2007. Guidelines for performing systematic literature reviews in software engineering. Technical Report. Technical report, ver. 2.3 ebse technical report. ebse.
Satwik Ram Kodandaram, Utku Uckun, Xiaojun Bi, IV Ramakrishnan, and Vikas Ashok. 2024. Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (St. John’s, NL, Canada) (ASSETS ’24). Association for Computing Machinery, New York, NY, USA, Article 73, 14 pages. DOI: 10.1145/3663548.3675605
Rand Koualty, Nien-Ying Chou, and Suleiman Alabdallah. 2024. Generative AI Agents, Build a Multilingual ChatGPT-based Customer Service Chatbot. In 2024 2nd International Conference on Foundation and Large Language Models (FLLM). 5–10. DOI: 10.1109/FLLM63129.2024.10852436
Felix Kretzer, Kristian Kolthoff, Christian Bartelt, Simone Paolo Ponzetto, and Alexander Maedche. 2025. Closing the loop between user stories and GUI prototypes: an LLM-based assistant for cross-functional integration in software development. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19.
Emily Kuang, Minghao Li, Mingming Fan, and Kristen Shinohara. 2024. Enhancing UX Evaluation Through Collaboration with Conversational AI Assistants: Effects of Proactive Dialogue and Timing. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24). Association for Computing Machinery, Article 3, 16 pages.
Geonsun Lee, Yue Yang, Jennifer Healey, and Dinesh Manocha. 2025. Since U Been Gone: Augmenting Context-Aware Transcriptions for Re-Engaging in Immersive VR Meetings. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20.
Geonsun Lee, Yue Yang, Jennifer Healey, and Dinesh Manocha. 2025. Since U Been Gone: Augmenting Context-Aware Transcriptions for Re-Engaging in Immersive VR Meetings. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 785, 20 pages. DOI: 10.1145/3706598.3714078
Seonghee Lee, Maho Kohga, Steve Landau, Sile O’Modhrain, and Hari Subramonyam. 2024. AltCanvas: a tile-based editor for visual content creation with generative AI for blind or visually impaired people. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility. 1–22.
Chaoyu Li, Sid Padmanabhuni, Maryam S Cheema, Hasti Seifi, and Pooyan Fazli. 2025. VideoA11y: Method and Dataset for Accessible Video Description. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 1055, 29 pages. DOI: 10.1145/3706598.3714096
Bingyang Liu, Haoyi Zhang, Xiaohan Gao, Zichen Kong, Xiyuan Tang, Yibo Lin, Runsheng Wang, and Ru Huang. 2025. LayoutCopilot: An LLM-Powered Multiagent Collaborative Framework for Interactive Analog Layout Design. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 44, 8 (2025), 3126–3139. DOI: 10.1109/TCAD.2025.3529805
Michael Xieyang Liu, Savvas Petridis, Vivian Tsai, Alexander J Fiannaca, Alex Olwal, Michael Terry, and Carrie J Cai. 2025. Gensors: Authoring Personalized Visual Sensors with Multimodal Foundation Models and Reasoning. In Proceedings of the 30th International Conference on Intelligent User Interfaces. 755–770.
Xinyi Lu, Simin Fan, Jessica Houghton, Lu Wang, and Xu Wang. 2023. ReadingQuizMaker: A Human-NLP Collaborative System that Supports Instructors to Design High-Quality Reading Quiz Questions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 454, 18 pages. DOI: 10.1145/3544548.3580957
Matteo Manzoni, Sergio Mascetti, Dragan Ahmetovic, Ryan Crabb, and James M. Coughlan. 2025. MapIO: A Gestural and Conversational Interface for Tactile Maps. IEEE Access 13 (2025), 84038–84056. DOI: 10.1109/ACCESS.2025.3566286
Yihan Mei, ZhaoWu, Junnan Yu,Wenan Li, and Zhibin Zhou. 2025. GeneyMAP: Exploring the Potential of GenAI to Facilitate Mapping User Journeys for UX Design. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–22.
Aditi Mishra, Frederik Brudy, Qian Zhou, George Fitzmaurice, and Fraser Anderson. 2025. WhatIF: Branched Narrative Fiction Visualization for Authoring Emergent Narratives using Large Language Models. In Proceedings of the 2025 Conference on Creativity and Cognition. 590–605.
Aditi Mishra, Frederik Brudy, Qian Zhou, George Fitzmaurice, and Fraser Anderson. 2025. WhatIF: Branched Narrative Fiction Visualization for Authoring Emergent Narratives using Large Language Models. In Proceedings of the 2025 Conference on Creativity and Cognition (C&C ’25). ACM, 590–605.
Peya Mowar, Yi-Hao Peng, Jason Wu, Aaron Steinfeld, and Jeffrey P Bigham. 2025. CodeA11y: Making AI Coding Assistants Useful for Accessible Web Development. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 45, 15 pages. DOI: 10.1145/3706598.3713335
Geetika Munjal, Austin Dious, Devansh Agarwal, Rahul Sharma, and Gunjan Jarawat. 2025. Blindviz—Intelligent Navigation and Environmental Awareness for the Visually Impaired. In 2025 3rd International Conference on Disruptive Technologies (ICDT). IEEE, 492–496.
Kálmán Balázs Neszlényi, Alex Milos, and Attila Kiss. 2024. AssistantGPT: Enhancing User Interaction with LLM Integration. In 2024 IEEE 22nd Jubilee International Symposium on Intelligent Systems and Informatics (SISY). IEEE, 000619–000624.
Shi Hui Ng, Lay-Ki Soon, and Tin Tin Su. 2023. Emotion-Aware Chatbot with Cultural Adaptation for Mitigating Work-Related Stress. In Proceedings of the Asian HCI Symposium 2023 (Online, Indonesia) (Asian CHI ’23). ACM, 41–50.
Jakob Nielsen. 1994. Usability engineering. Morgan Kaufmann.
Zheng Ning, Brianna L Wimer, Kaiwen Jiang, Keyi Chen, Jerrick Ban, Yapeng Tian, Yuhang Zhao, and Toby Jia-Jun Li. 2024. SPICA: interactive video content exploration through augmented audio descriptions for blind or low-vision viewers. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–18.
Ziqi Pan, Xiucheng Zhang, Zisu Li, Zhenhui Peng, Mingming Fan, and Xiaojuan Ma. 2025. ACKnowledge: A Computational Framework for Human Compatible Affordance-based Interaction Planning in Real-world Contexts. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20.
Rock Yuren Pang, Hope Schroeder, Kynnedy Simone Smith, Solon Barocas, Ziang Xiao, Emily Tseng, and Danielle Bragg. 2025. Understanding the LLMification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 456, 20 pages. DOI: 10.1145/3706598.3713726
Jimin Park, Chaerin Lee, Eunbin Cho, and Uran Oh. 2024. Enhancing the Podcast Browsing Experience through Topic Segmentation and Visualization with Generative AI. In Proceedings of the 2024 ACM International Conference on Interactive Media Experiences (Stockholm, Sweden) (IMX ’24). Association for Computing Machinery, 117–128.
Jaehyeong Park, Juncheol Ye, Seungkook Lee, Hyun W. Ka, and Dongsu Han. 2025. NarrAD: Automatic Generation of Audio Descriptions for Movies with Rich Narrative Context. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 409–419.
Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and software technology 64 (2015), 1–18. DOI: 10.1016/j.infsof.2015.03.007
Mauro Pezzè, Silvia Abrahão, Birgit Penzenstadler, Denys Poshyvanyk, Abhik Roychoudhury, and Tao Yue. 2025. A 2030 Roadmap for Software Engineering. ACM Trans. Softw. Eng. Methodol. 34, 5, Article 118 (may 2025), 55 pages. DOI: 10.1145/3731559
Wasifur Rahman, Abdelrahman Abdelkader, Sangwu Lee, Phillip Yang, Md Saiful Islam, Tariq Adnan, Masum Hasan, Ellen Wagner, Sooyong Park, E. Ray Dorsey, Catherine Schwartz, Karen Jaffe, and Ehsan Hoque. 2024. A User-Centered Framework to Empower People with Parkinson’s Disease. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 7, 4, Article 175 (Jan. 2024), 29 pages. DOI: 10.1145/3631430
Shwetha Rajaram, Hemant Bhaskar Surale, Codie McConkey, Carine Rognon, Hrim Mehta, Michael Glueck, and Christopher Collins. 2025. Gesture and Audio-Haptic Guidance Techniques to Direct Conversations with Intelligent Voice Interfaces. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20.
Mashrur Rashik, Shilpa Sweth, Nishtha Agrawal, Saiyyam Kochar, Kara M Smith, Fateme Rajabiyazdi, Vidya Setlur, Narges Mahyar, and Ali Sarvghad. 2025. AI-Enabled Conversational Journaling for Advancing Parkinson’s Disease Symptom Tracking. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). ACM, Article 1029, 23 pages.
Marcos Ribeiro, Joana Santos, João Lobo, Sílvia Araújo, Luís Magalhães, and Telmo Adão. 2024. VR, AR, gamification and AI towards the next generation of systems supporting cultural heritage: addressing challenges of a museum context. In Proceedings of the 29th International ACM Conference on 3D Web Technology (Guimarães, Portugal) (Web3D ’24). Association for Computing Machinery, New York, NY, USA, Article 9, 10 pages. DOI: 10.1145/3665318.3677172
Nathalie Riche, Anna Offenwanger, Frederic Gmeiner, David Brown, Hugo Romat, Michel Pahud, Nicolai Marquardt, Kori Inkpen, and Ken Hinckley. 2025. AI-Instruments: Embodying Prompts as Instruments to Abstract&Reflect Graphical Interface Commands as General-Purpose Tools. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, Article 1104, 18 pages.
Y. Rogers, H. Sharp, and J. Preece. 2013. Design de Interação - Além da Interação Humano-Computador (3th. ed.). Bookman Editora.
Phillip Schneider, Nils Rehtanz, Kristiina Jokinen, and Florian Matthes. 2023. Voice-Based Conversational Agents and Knowledge Graphs for Improving News Search in Assisted Living. In Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments (Corfu, Greece) (PETRA ’23). Association for Computing Machinery, New York, NY, USA, 645–651. DOI: 10.1145/3594806.3596534
Chirag Shah, Ryen White, Reid Andersen, Georg Buscher, Scott Counts, Sarkar Das, Ali Montazer, Sathish Manivannan, Jennifer Neville, Nagu Rangan, et al. 2023. Using large language models to generate, validate, and apply user intent taxonomies. ACM Transactions on the Web (2023).
Chirag Shah, Ryen White, Reid Andersen, Georg Buscher, Scott Counts, Sarkar Das, Ali Montazer, Sathish Manivannan, Jennifer Neville, Nagu Rangan, Tara Safavi, Siddharth Suri, Mengting Wan, Leijie Wang, and Longqi Yang. 2025. Using Large Language Models to Generate, Validate, and Apply User Intent Taxonomies. ACM Trans. Web (May 2025). DOI: 10.1145/3732294 Just Accepted.
Hanshu Shen, Lyukesheng Shen, Wenqi Wu, and Kejun Zhang. 2025. Ideationweb: Tracking the evolution of design ideas in human-ai co-creation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19.
Xinyu Shi, Yinghou Wang, Ryan Rossi, and Jian Zhao. 2025. Brickify: Enabling Expressive Design Intent Specification through Direct Manipulation on Design Tokens. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20.
Momin N Siddiqui, Roy D Pea, and Hari Subramonyam. 2025. Script&Shift: A layered interface paradigm for integrating content development and rhetorical strategy with llm writing assistants. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19.
Tianyu Song, Zhengyi Liu, Ruibin Zhao, and Jie Fu. 2024. ElderEase AR: Enhancing Elderly Daily Living with the Multimodal Large Language Model and Augmented Reality. In Proceedings of the 2024 International Conference on Virtual Reality Technology. 60–67.
Sangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li, and Haijun Xia. 2024. Luminate: Structured generation and exploration of design space with large language models for human-ai co-creation. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–26.
Xin Sun, Yunjie Liu, Jos A. Bosch, and Zhuying Li. 2025. Interface Matters: Exploring Human Trust in Health Information from Large Language Models via Text, Speech, and Embodiment. Proc. ACM Hum.-Comput. Interact. 9, 2, Article CSCW116 (May 2025), 32 pages.
Macy Takaffoli, Sijia Li, and Ville Mäkelä. 2024. Generative AI in User Experience Design and Research: How Do UX Practitioners, Teams, and Companies Use GenAI in Industry?. In Proceedings of the 2024 ACM Designing Interactive Systems Conference (Copenhagen, Denmark) (DIS ’24). Association for Computing Machinery, New York, NY, USA, 1579–1593. DOI: 10.1145/3643834.3660720
Michael Tamascelli, Olivia Bunch, Blake Fowler, Maryam Taeb, and Achraf Cohen. 2025. Academic Advising Chatbot Powered with AI Agent. In Proceedings of the 2025 ACM Southeast Conference. 195–202.
Xilin Tang, Jerrod Windham, and Benjamin Bush. 2024. Pre-AI and post-AI design: balancing human Creativity and AI Tools in the Industrial Design Process. In Proceeding of the 2024 International Conference on Artificial Intelligence and Future Education. 100–108.
Kelsey Turbeville, Jennarong Muengtaweepongsa, Samuel Stevens, Jason Moss, Amy Pon, Kyra Lee, Charu Mehra, Jenny Gutierrez Villalobos, and Ranjitha Kumar. 2024. LLM-powered Multimodal Insight Summarization for UX Testing. In Proceedings of the 26th International Conference on Multimodal Interaction (ICMI ’24). Association for Computing Machinery, 4–11.
Zhan Wang, Lin-Ping Yuan, Liangwei Wang, Bingchuan Jiang, and Wei Zeng. 2024. VirtuWander: Enhancing Multi-modal Interaction for Virtual Tour Guidance through Large Language Models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 612, 20 pages. DOI: 10.1145/3613904.3642235
Zijie J Wang, Chinmay Kulkarni, Lauren Wilcox, Michael Terry, and Michael Madaio. 2024. Farsight: Fostering responsible ai awareness during ai application prototyping. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–40.
Tobias M Weinberg, Kowe Kadoma, Ricardo E. Gonzalez Penuela, Stephanie Valencia, and Thijs Roumen. 2025. Why So Serious? Exploring Timely Humorous Comments in AAC Through AI-Powered Interfaces. In Proceedings of the 2025 CHI (CHI ’25). ACM, Article 759, 19 pages.
Justin D. Weisz, Jessica He, Michael Muller, Gabriela Hoefer, Rachel Miles, and Werner Geyer. 2024. Design Principles for Generative AI Applications. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 378, 22 pages. DOI: 10.1145/3613904.3642466
Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022. Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In Proceedings of the 2022 CHI conference on human factors in computing systems. 1–22.
Haijun Xia, Tony Wang, Aditya Gunturu, Peiling Jiang, William Duan, and Xiaoshuo Yao. 2023. CrossTalk: Intelligent substrates for language-oriented interaction in video-based communication and collaboration. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–16.
Wei Xiang, Hanfei Zhu, Suqi Lou, Xinli Chen, Zhenghua Pan, Yuping Jin, Shi Chen, and Lingyun Sun. 2024. SimUser: Generating Usability Feedback by Simulating Various Users Interacting with Mobile Applications. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 9, 17 pages. DOI: 10.1145/3613904.3642481
Shuchang Xu, Chang Chen, Zichen Liu, Xiaofu Jin, Lin-Ping Yuan, Yukang Yan, and Huamin Qu. 2024. Memory Reviver: Supporting Photo-Collection Reminiscence for People with Visual Impairment via a Proactive Chatbot. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. 1–17.
Litao Yan, Alyssa Hwang, Zhiyuan Wu, and Andrew Head. 2024. Ivie: Lightweight Anchored Explanations of Just-Generated Code. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 140, 15 pages. DOI: 10.1145/3613904.3642239
Sojeong Yun and Youn-kyung Lim. 2025. User Experience with LLM-powered Conversational Recommendation Systems: A Case of Music Recommendation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 898, 15 pages. DOI: 10.1145/3706598.3713347
Guanhua Zhang, Mohamed Adel Naguib Ahmed, Zhiming Hu, and Andreas Bulling. 2025. Summact: Uncovering user intentions through interactive behaviour summarisation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–17.
Mengxi Zhang, Huaxiao Liu, Changhao Du, Tengmei Wang, Han Li, Pei Huang, and Chunyang Chen. 2025. Distinguishing GUI Component States for Blind Users using Large Language Models. ACM Transactions on Software Engineering and Methodology (2025).
Tianhao Zhang, Fu Peiguo, Jie Liu, Yihe Zhang, and Xingmei Chen. 2024. NLDesign: A UI Design Tool for Natural Language Interfaces. In Proceedings of the ACM Turing Award Celebration Conference - China 2024 (Changsha, China) (ACM-TURC ’24). Association for Computing Machinery, New York, NY, USA, 153–158. DOI: 10.1145/3674399.3674455
Qingxiao Zheng, Minrui Chen, Pranav Sharma, Yiliu Tang, Mehul Oswal, Yiren Liu, and Yun Huang. 2025. EvAlignUX: Advancing UX Evaluation through LLM-Supported Metrics Exploration. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). ACM, Article 1051, 25 pages.
Mingyuan Zhong, Ruolin Chen, Xia Chen, James Fogarty, and Jacob OWobbrock. 2025. ScreenAudit: Detecting Screen Reader Accessibility Errors in Mobile Apps Using Large Language Models. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19.
Ting Zhou, Yanjie Zhao, Xinyi Hou, Xiaoyu Sun, Kai Chen, and Haoyu Wang. 2025. DeclarUI: Bridging Design and Development with Automated Declarative UI Code Generation. Proc. ACM Softw. Eng. 2, FSE, Article FSE011 (June 2025), 23 pages.
Wajdi Aljedaani, Abdulrahman Habib, Ahmed Aljohani, Marcelo Eler, and Yunhe Feng. 2024. Does ChatGPT Generate Accessible Code? Investigating Accessibility Challenges in LLM-Generated Source Code. In Proceedings of the 21st International Web for All Conference (Singapore, Singapore) (W4A ’24). Association for Computing Machinery, New York, NY, USA, 165–176. DOI: 10.1145/3677846.3677854
Rubén Alonso, Danilo Dessí, Antonello Meloni, Marco Murgia, Diego Reforgiato Recupero, and Giuseppe Scarpi. 2024. A Seamless ChatGPT Knowledge Plug-in for the Labour Market. IEEE Access 12 (2024), 155821–155837. DOI: 10.1109/ACCESS.2024.3485111
Riku Arakawa, Jill Fain Lehman, and Mayank Goel. 2024. Prism-q&a: Step-aware voice assistant on a smartwatch enabled by multimodal procedure tracking and large language models. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 4 (2024), 1–26.
Aram Bahrini, Mohammadsadra Khamoshifar, Hossein Abbasimehr, Robert J. Riggs, Maryam Esmaeili, Rastin Mastali Majdabadkohne, and Morteza Pasehvar. 2023. ChatGPT: Applications, Opportunities, and Threats. In 2023 SIEDS. 274– 279. DOI: 10.1109/SIEDS58326.2023.10137850
Jose Barambones, Cristian Moral, Angélica de Antonio, Ricardo Imbert, Loïc Martínez-Normand, and Elena Villalba-Mora. 2024. ChatGPT for Learning HCI Techniques: A Case Study on Interviews for Personas. IEEE Transactions on Learning Technologies 17 (2024), 1460–1475. DOI: 10.1109/TLT.2024..3386095
Simone Barbosa and Bruno Silva. 2010. Interação humano-computador. Elsevier Brasil.
Majid Behravan and Denis Gračanin. 2025. From Voices to Worlds: Developing an AI-Powered Framework for 3D Object Generation in Augmented Reality. In 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 150–155.
Benyamin Beyzaei, Saghar Talebipour, Ghazal Rafiei, Nenad Medvidović, and Sam Malek. 2025. Automated Test Transfer across Android Apps using Large Language Models. Proc. ACM Softw. Eng. 2, ISSTA, Article ISSTA098 (June 2025), 24 pages.
Anastasiia Birillo, Elizaveta Artser, Anna Potriasaeva, Ilya Vlasov, Katsiaryna Dzialets, Yaroslav Golubev, Igor Gerasimov, Hieke Keuning, and Timofey Bryksin. 2024. One step at a time: Combining llms and static analysis to generate next-step hints for programming tasks. In Proceedings of the 24th Koli Calling International Conference on Computing Education Research. 1–12.
Mariana Gomes Borges, Claiton Marques Correa, and Milene Selbach Silveira. 2024. Human+ Human or Human+ AI: A Comparative Analysis of a Narrative Visualization Design Guide. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems. 1–13.
Paul Brie, Nicolas Burny, Arthur Sluÿters, and Jean Vanderdonckt. 2023. Evaluating a Large Language Model on Searching for GUI Layouts. Proc. ACM Hum.-Comput. Interact. 7, EICS, Article 178 (June 2023), 37 pages. DOI: 10.1145/3593230
Yining Cao, Peiling Jiang, and Haijun Xia. 2025. Generative and Malleable User Interfaces with Generative and Evolving Task-Driven Data Model. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20.
Inês Carmo, Patrícia Costa, and Pedro Santana. 2024. Boosting children’s reading motivation with LLM-generated story crossovers. In 2024 International Conference on Graphics and Interaction (ICGI). IEEE, 1–8.
Chaoran Chen, Weijun Li, Wenxin Song, Yanfang Ye, Yaxing Yao, and Toby Jia-Jun Li. 2024. An empathy-based sandbox approach to bridge the privacy gap among attitudes, goals, knowledge, and behaviors. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–28.
Chen Chen, Cuong Nguyen, Thibault Groueix, Vladimir G Kim, and Nadir Weibel. 2024. Memovis: A GenAI-powered tool for creating companion reference images for 3d design feedback. ACM Transactions on Computer-Human Interaction 31, 5 (2024), 1–41.
Liuqing Chen, Zhaojun Jiang, Duowei Xia, Zebin Cai, Lingyun Sun, Peter Childs, and Haoyu Zuo. 2024. BIDTrainer: An LLMs-driven Education Tool for Enhancing the Understanding and Reasoning in Bio-inspired Design. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 676, 20 pages. DOI: 10.1145/3613904.3642887
Pei Chen, Jiayi Yao, Zhuoyi Cheng, Yichen Cai, Jiayang Li, Weitao You, and Lingyun Sun. 2025. CoExploreDS: Framing and Advancing Collaborative Design Space Exploration Between Human and AI. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 22, 20 pages. DOI: .org/10.1145/3706598.3713869
Sharfuddin Khan Chisty, Anika Tahsin Miami, and Jannatun Noor. 2025. Smart Sheba: Enhancing Elderly User Experience with LLM-Enabled Chatbots and User-Centered Design. In Proceedings of the 13th International Conference on Information & Communication Technologies and Development (Nairobi, Kenya) (ICTD ’24). Association for Computing Machinery, New York, NY, USA, 69–83. DOI: 10.1145/3700794.3700803
Sravan Kumar Chittimalla and Leela Krishna M Potluri. 2025. Enhancing Yoga Pose Recognition Using Large Language Models with Real-World Dataset Integration for Personalized Fitness Assistance. In 2024 International Conference on IT Innovation and Knowledge Discovery (ITIKD). 1–6. DOI: 10.1109/ITIKD63574.2025.11004828
Nicholas Davis, Jacob Sherson, and Janet Rafner. 2025. The Co-Creative Design Framework for Hybrid Intelligence. In Proceedings of the 2025 Conference on Creativity and Cognition (C&C ’25). Association for Computing Machinery, New York, NY, USA, 560–572. DOI: 10.1145/3698061.3726934
Gabriel Dengler, Peter Bazan, Reinhard German, Pooia Lalbakhsh, and Ariel Liebmann. 2024. A Conversational Human-Computer Interface for Smart Energy System Simulation Environments. In Proceedings of the Winter Simulation Conference (San Antonio, Texas, USA) (WSC ’23). IEEE Press, 2978–2989.
Ati Suci Dian Martha, Sri Widowati, Desy Puspa Rahayu, Muhammad Ilham Nursyawal, and Rayhan Rizqial Hariz. 2024. Examining Usability and Student Experience of ChatGPT as a Learning Tool on Moodle in Higher Education. In 2024 International Conference on Information Technology Systems and Innovation (ICITSI). 8–13. DOI: 10.1109/ICITSI65188.2024.10929431
Hyo Jin Do, Michelle Brachman, Casey Dugan, James M Johnson, Julia Lauer, Priyanshu Rai, and Qian Pan. 2024. Grounding with Structure: Exploring Design Variations of Grounded Human-AI Collaboration in a Natural Language Interface. Proceedings of the ACM on Human-Computer Interaction 8, CSCW2 (2024), 1–27.
Peitong Duan, JeremyWarner, Yang Li, and Bjoern Hartmann. 2024. Generating Automatic Feedback on UI Mockups with Large Language Models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 6, 20 pages. DOI: 10.1145/3613904.3642782
Fernanda Espinoza, Darren Cook, Chris R Butler, and Rafael A Calvo. 2023. Supporting dementia caregivers in Peru through chatbots: generative AI vs structured conversations. In Electronic Workshops in Computing. BCS Learning & Development, 89–98. DOI: 10.14236/ewic/BCSHCI2023.11
K. J. Kevin Feng, Q. Vera Liao, Ziang Xiao, Jennifer Wortman Vaughan, Amy X. Zhang, and David W. McDonald. 2025. Canvil: Designerly Adaptation for LLMPowered User Experiences. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, Article 932, 22 pages.
Victoria Firsanova. 2023. Towards Building a Mobile App for People on the Spectrum. In Companion Proceedings of the ACM Web Conference 2023 (Austin, TX, USA) (WWW ’23 Companion). Association for Computing Machinery, New York, NY, USA, 555–559. DOI: 10.1145/3543873.3587533
Claudia Flores-Saviaga, Benjamin V. Hanrahan, Kashif Imteyaz, Steven Clarke, and Saiph Savage*. 2025. The Impact of Generative AI Coding Assistants on Developers Who Are Visually Impaired. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 1164, 17 pages. DOI: 10.1145/3706598.3714008
Toru Furusawa. 2025. In-Vehicle Artificial Intelligence Agent System with Local Large Language Models for Agent Selection and Vehicle Control. In 2025 International Conference on Computing, Networking and Communications (ICNC). IEEE, 259–264.
Muhammet Kürşat Görmez, Murat Yılmaz, and Paul M Clarke. 2024. Large language models for software engineering: A systematic mapping study. In European Conference on Software Process Improvement, Vol. 2179. Springer, Cham, 64–79. DOI: 10.1007/978-3-031-71139-8_5
Joshua Gorniak, Yoon Kim, Donglai Wei, and Nam Wook Kim. 2024. VizAbility: Enhancing Chart Accessibility with LLM-based Conversational Interaction. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (Pittsburgh, PA, USA) (UIST ’24). Article 89, 19 pages.
Yi Guo, Danqing Shi, Mingjuan Guo, YanqiuWu, Nan Cao, and Qing Chen. 2024. Talk2Data: A Natural Language Interface for Exploratory Visual Analysis via Question Decomposition. ACM Trans. Interact. Intell. Syst. 14, 2, Article 8 (April 2024), 24 pages. DOI: 10.1145/3643894
Jihyeong Hong, Yokyung Lee, Dae Hyun Kim, DaEun Choi, Yeo-Jin Yoon, Gyucheol Lee, Zucheul Lee, and Juho Kim. 2024. A context-aware onboarding agent for metaverse powered by large language models. In Proceedings of the 2024 ACM Designing Interactive Systems Conference. 1857–1874.
Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large Language Models for Software Engineering: A Systematic Literature Review. ACM Trans. Softw. Eng. Methodol. 33, 8, Article 220 (Dec. 2024), 79 pages. DOI: 10.1145/3695988
Forrest Huang, Gang Li, Tao Li, and Yang Li. 2024. Automatic Macro Mining from Interaction Traces at Scale. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 1038, 16 pages. DOI: 10.1145/3613904.3642074
Faria Huq, Abdus Samee, David Chuan-En Lin, Alice Xiaodi Tang, and Jeffrey P Bigham. 2025. NoTeeline: Supporting Real-Time, Personalized Notetaking with LLM-Enhanced Micronotes. In Proceedings of the 30th International Conference on Intelligent User Interfaces. 1064–1081.
Srujana Inturi, Swarna Latha Potluri, Guda Vanitha, and B Indira. 2025. VirtualMe: A Large Language Model-Powered Chatbot for Enhanced Human Computer Interaction. In 2025 International Conference on Machine Learning and Autonomous Systems (ICMLAS). IEEE, 626–634.
Dhiraj Jadhav, Sakshee Agrawal, Sakshi Jagdale, Pranav Salunkhe, and Raj Salunkhe. 2024. AI-Driven Text-to-Multimedia Content Generation: Enhancing Modern Content Creation. In 2024 8th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC). 1610–1615. DOI: 10.1109/ISMAC61858.2024.10714771
Jazette Johnson, Hira Jamshed, Rachael Zuppke, Amanda Leggett, Emily Mower Provost, and Robin N. Brewer. 2025. “I felt listened to”: Evaluating an AI-Powered Reflection Tool for Care Partners. ACM Trans. Access. Comput. (April 2025). Just Accepted.
Aayush Kapoor and Sujala D. Shetty. 2024. Enhancing Healthcare Information Accessibility Through a Generative Medical Chatbot. In 2024 ICETCS. 1–4.
Kai Karren, Michael Schmitz, and Stefan Schaffer. 2024. Improving conversational user interfaces for citizen complaint management through enhanced contextual feedback. In Proceedings of the 6th ACM Conference on Conversational User Interfaces. 1–11.
Yewon Kim, Sung-Ju Lee, and Chris Donahue. 2025. Amuse: Human-AI Collaborative Songwriting with Multimodal Inspirations. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–28.
Barbara Kitchenham, Stuart Charters, et al. 2007. Guidelines for performing systematic literature reviews in software engineering. Technical Report. Technical report, ver. 2.3 ebse technical report. ebse.
Satwik Ram Kodandaram, Utku Uckun, Xiaojun Bi, IV Ramakrishnan, and Vikas Ashok. 2024. Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (St. John’s, NL, Canada) (ASSETS ’24). Association for Computing Machinery, New York, NY, USA, Article 73, 14 pages. DOI: 10.1145/3663548.3675605
Rand Koualty, Nien-Ying Chou, and Suleiman Alabdallah. 2024. Generative AI Agents, Build a Multilingual ChatGPT-based Customer Service Chatbot. In 2024 2nd International Conference on Foundation and Large Language Models (FLLM). 5–10. DOI: 10.1109/FLLM63129.2024.10852436
Felix Kretzer, Kristian Kolthoff, Christian Bartelt, Simone Paolo Ponzetto, and Alexander Maedche. 2025. Closing the loop between user stories and GUI prototypes: an LLM-based assistant for cross-functional integration in software development. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19.
Emily Kuang, Minghao Li, Mingming Fan, and Kristen Shinohara. 2024. Enhancing UX Evaluation Through Collaboration with Conversational AI Assistants: Effects of Proactive Dialogue and Timing. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24). Association for Computing Machinery, Article 3, 16 pages.
Geonsun Lee, Yue Yang, Jennifer Healey, and Dinesh Manocha. 2025. Since U Been Gone: Augmenting Context-Aware Transcriptions for Re-Engaging in Immersive VR Meetings. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20.
Geonsun Lee, Yue Yang, Jennifer Healey, and Dinesh Manocha. 2025. Since U Been Gone: Augmenting Context-Aware Transcriptions for Re-Engaging in Immersive VR Meetings. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 785, 20 pages. DOI: 10.1145/3706598.3714078
Seonghee Lee, Maho Kohga, Steve Landau, Sile O’Modhrain, and Hari Subramonyam. 2024. AltCanvas: a tile-based editor for visual content creation with generative AI for blind or visually impaired people. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility. 1–22.
Chaoyu Li, Sid Padmanabhuni, Maryam S Cheema, Hasti Seifi, and Pooyan Fazli. 2025. VideoA11y: Method and Dataset for Accessible Video Description. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 1055, 29 pages. DOI: 10.1145/3706598.3714096
Bingyang Liu, Haoyi Zhang, Xiaohan Gao, Zichen Kong, Xiyuan Tang, Yibo Lin, Runsheng Wang, and Ru Huang. 2025. LayoutCopilot: An LLM-Powered Multiagent Collaborative Framework for Interactive Analog Layout Design. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 44, 8 (2025), 3126–3139. DOI: 10.1109/TCAD.2025.3529805
Michael Xieyang Liu, Savvas Petridis, Vivian Tsai, Alexander J Fiannaca, Alex Olwal, Michael Terry, and Carrie J Cai. 2025. Gensors: Authoring Personalized Visual Sensors with Multimodal Foundation Models and Reasoning. In Proceedings of the 30th International Conference on Intelligent User Interfaces. 755–770.
Xinyi Lu, Simin Fan, Jessica Houghton, Lu Wang, and Xu Wang. 2023. ReadingQuizMaker: A Human-NLP Collaborative System that Supports Instructors to Design High-Quality Reading Quiz Questions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 454, 18 pages. DOI: 10.1145/3544548.3580957
Matteo Manzoni, Sergio Mascetti, Dragan Ahmetovic, Ryan Crabb, and James M. Coughlan. 2025. MapIO: A Gestural and Conversational Interface for Tactile Maps. IEEE Access 13 (2025), 84038–84056. DOI: 10.1109/ACCESS.2025.3566286
Yihan Mei, ZhaoWu, Junnan Yu,Wenan Li, and Zhibin Zhou. 2025. GeneyMAP: Exploring the Potential of GenAI to Facilitate Mapping User Journeys for UX Design. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–22.
Aditi Mishra, Frederik Brudy, Qian Zhou, George Fitzmaurice, and Fraser Anderson. 2025. WhatIF: Branched Narrative Fiction Visualization for Authoring Emergent Narratives using Large Language Models. In Proceedings of the 2025 Conference on Creativity and Cognition. 590–605.
Aditi Mishra, Frederik Brudy, Qian Zhou, George Fitzmaurice, and Fraser Anderson. 2025. WhatIF: Branched Narrative Fiction Visualization for Authoring Emergent Narratives using Large Language Models. In Proceedings of the 2025 Conference on Creativity and Cognition (C&C ’25). ACM, 590–605.
Peya Mowar, Yi-Hao Peng, Jason Wu, Aaron Steinfeld, and Jeffrey P Bigham. 2025. CodeA11y: Making AI Coding Assistants Useful for Accessible Web Development. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 45, 15 pages. DOI: 10.1145/3706598.3713335
Geetika Munjal, Austin Dious, Devansh Agarwal, Rahul Sharma, and Gunjan Jarawat. 2025. Blindviz—Intelligent Navigation and Environmental Awareness for the Visually Impaired. In 2025 3rd International Conference on Disruptive Technologies (ICDT). IEEE, 492–496.
Kálmán Balázs Neszlényi, Alex Milos, and Attila Kiss. 2024. AssistantGPT: Enhancing User Interaction with LLM Integration. In 2024 IEEE 22nd Jubilee International Symposium on Intelligent Systems and Informatics (SISY). IEEE, 000619–000624.
Shi Hui Ng, Lay-Ki Soon, and Tin Tin Su. 2023. Emotion-Aware Chatbot with Cultural Adaptation for Mitigating Work-Related Stress. In Proceedings of the Asian HCI Symposium 2023 (Online, Indonesia) (Asian CHI ’23). ACM, 41–50.
Jakob Nielsen. 1994. Usability engineering. Morgan Kaufmann.
Zheng Ning, Brianna L Wimer, Kaiwen Jiang, Keyi Chen, Jerrick Ban, Yapeng Tian, Yuhang Zhao, and Toby Jia-Jun Li. 2024. SPICA: interactive video content exploration through augmented audio descriptions for blind or low-vision viewers. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–18.
Ziqi Pan, Xiucheng Zhang, Zisu Li, Zhenhui Peng, Mingming Fan, and Xiaojuan Ma. 2025. ACKnowledge: A Computational Framework for Human Compatible Affordance-based Interaction Planning in Real-world Contexts. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20.
Rock Yuren Pang, Hope Schroeder, Kynnedy Simone Smith, Solon Barocas, Ziang Xiao, Emily Tseng, and Danielle Bragg. 2025. Understanding the LLMification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 456, 20 pages. DOI: 10.1145/3706598.3713726
Jimin Park, Chaerin Lee, Eunbin Cho, and Uran Oh. 2024. Enhancing the Podcast Browsing Experience through Topic Segmentation and Visualization with Generative AI. In Proceedings of the 2024 ACM International Conference on Interactive Media Experiences (Stockholm, Sweden) (IMX ’24). Association for Computing Machinery, 117–128.
Jaehyeong Park, Juncheol Ye, Seungkook Lee, Hyun W. Ka, and Dongsu Han. 2025. NarrAD: Automatic Generation of Audio Descriptions for Movies with Rich Narrative Context. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 409–419.
Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and software technology 64 (2015), 1–18. DOI: 10.1016/j.infsof.2015.03.007
Mauro Pezzè, Silvia Abrahão, Birgit Penzenstadler, Denys Poshyvanyk, Abhik Roychoudhury, and Tao Yue. 2025. A 2030 Roadmap for Software Engineering. ACM Trans. Softw. Eng. Methodol. 34, 5, Article 118 (may 2025), 55 pages. DOI: 10.1145/3731559
Wasifur Rahman, Abdelrahman Abdelkader, Sangwu Lee, Phillip Yang, Md Saiful Islam, Tariq Adnan, Masum Hasan, Ellen Wagner, Sooyong Park, E. Ray Dorsey, Catherine Schwartz, Karen Jaffe, and Ehsan Hoque. 2024. A User-Centered Framework to Empower People with Parkinson’s Disease. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 7, 4, Article 175 (Jan. 2024), 29 pages. DOI: 10.1145/3631430
Shwetha Rajaram, Hemant Bhaskar Surale, Codie McConkey, Carine Rognon, Hrim Mehta, Michael Glueck, and Christopher Collins. 2025. Gesture and Audio-Haptic Guidance Techniques to Direct Conversations with Intelligent Voice Interfaces. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20.
Mashrur Rashik, Shilpa Sweth, Nishtha Agrawal, Saiyyam Kochar, Kara M Smith, Fateme Rajabiyazdi, Vidya Setlur, Narges Mahyar, and Ali Sarvghad. 2025. AI-Enabled Conversational Journaling for Advancing Parkinson’s Disease Symptom Tracking. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). ACM, Article 1029, 23 pages.
Marcos Ribeiro, Joana Santos, João Lobo, Sílvia Araújo, Luís Magalhães, and Telmo Adão. 2024. VR, AR, gamification and AI towards the next generation of systems supporting cultural heritage: addressing challenges of a museum context. In Proceedings of the 29th International ACM Conference on 3D Web Technology (Guimarães, Portugal) (Web3D ’24). Association for Computing Machinery, New York, NY, USA, Article 9, 10 pages. DOI: 10.1145/3665318.3677172
Nathalie Riche, Anna Offenwanger, Frederic Gmeiner, David Brown, Hugo Romat, Michel Pahud, Nicolai Marquardt, Kori Inkpen, and Ken Hinckley. 2025. AI-Instruments: Embodying Prompts as Instruments to Abstract&Reflect Graphical Interface Commands as General-Purpose Tools. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, Article 1104, 18 pages.
Y. Rogers, H. Sharp, and J. Preece. 2013. Design de Interação - Além da Interação Humano-Computador (3th. ed.). Bookman Editora.
Phillip Schneider, Nils Rehtanz, Kristiina Jokinen, and Florian Matthes. 2023. Voice-Based Conversational Agents and Knowledge Graphs for Improving News Search in Assisted Living. In Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments (Corfu, Greece) (PETRA ’23). Association for Computing Machinery, New York, NY, USA, 645–651. DOI: 10.1145/3594806.3596534
Chirag Shah, Ryen White, Reid Andersen, Georg Buscher, Scott Counts, Sarkar Das, Ali Montazer, Sathish Manivannan, Jennifer Neville, Nagu Rangan, et al. 2023. Using large language models to generate, validate, and apply user intent taxonomies. ACM Transactions on the Web (2023).
Chirag Shah, Ryen White, Reid Andersen, Georg Buscher, Scott Counts, Sarkar Das, Ali Montazer, Sathish Manivannan, Jennifer Neville, Nagu Rangan, Tara Safavi, Siddharth Suri, Mengting Wan, Leijie Wang, and Longqi Yang. 2025. Using Large Language Models to Generate, Validate, and Apply User Intent Taxonomies. ACM Trans. Web (May 2025). DOI: 10.1145/3732294 Just Accepted.
Hanshu Shen, Lyukesheng Shen, Wenqi Wu, and Kejun Zhang. 2025. Ideationweb: Tracking the evolution of design ideas in human-ai co-creation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19.
Xinyu Shi, Yinghou Wang, Ryan Rossi, and Jian Zhao. 2025. Brickify: Enabling Expressive Design Intent Specification through Direct Manipulation on Design Tokens. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20.
Momin N Siddiqui, Roy D Pea, and Hari Subramonyam. 2025. Script&Shift: A layered interface paradigm for integrating content development and rhetorical strategy with llm writing assistants. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19.
Tianyu Song, Zhengyi Liu, Ruibin Zhao, and Jie Fu. 2024. ElderEase AR: Enhancing Elderly Daily Living with the Multimodal Large Language Model and Augmented Reality. In Proceedings of the 2024 International Conference on Virtual Reality Technology. 60–67.
Sangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li, and Haijun Xia. 2024. Luminate: Structured generation and exploration of design space with large language models for human-ai co-creation. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–26.
Xin Sun, Yunjie Liu, Jos A. Bosch, and Zhuying Li. 2025. Interface Matters: Exploring Human Trust in Health Information from Large Language Models via Text, Speech, and Embodiment. Proc. ACM Hum.-Comput. Interact. 9, 2, Article CSCW116 (May 2025), 32 pages.
Macy Takaffoli, Sijia Li, and Ville Mäkelä. 2024. Generative AI in User Experience Design and Research: How Do UX Practitioners, Teams, and Companies Use GenAI in Industry?. In Proceedings of the 2024 ACM Designing Interactive Systems Conference (Copenhagen, Denmark) (DIS ’24). Association for Computing Machinery, New York, NY, USA, 1579–1593. DOI: 10.1145/3643834.3660720
Michael Tamascelli, Olivia Bunch, Blake Fowler, Maryam Taeb, and Achraf Cohen. 2025. Academic Advising Chatbot Powered with AI Agent. In Proceedings of the 2025 ACM Southeast Conference. 195–202.
Xilin Tang, Jerrod Windham, and Benjamin Bush. 2024. Pre-AI and post-AI design: balancing human Creativity and AI Tools in the Industrial Design Process. In Proceeding of the 2024 International Conference on Artificial Intelligence and Future Education. 100–108.
Kelsey Turbeville, Jennarong Muengtaweepongsa, Samuel Stevens, Jason Moss, Amy Pon, Kyra Lee, Charu Mehra, Jenny Gutierrez Villalobos, and Ranjitha Kumar. 2024. LLM-powered Multimodal Insight Summarization for UX Testing. In Proceedings of the 26th International Conference on Multimodal Interaction (ICMI ’24). Association for Computing Machinery, 4–11.
Zhan Wang, Lin-Ping Yuan, Liangwei Wang, Bingchuan Jiang, and Wei Zeng. 2024. VirtuWander: Enhancing Multi-modal Interaction for Virtual Tour Guidance through Large Language Models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 612, 20 pages. DOI: 10.1145/3613904.3642235
Zijie J Wang, Chinmay Kulkarni, Lauren Wilcox, Michael Terry, and Michael Madaio. 2024. Farsight: Fostering responsible ai awareness during ai application prototyping. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–40.
Tobias M Weinberg, Kowe Kadoma, Ricardo E. Gonzalez Penuela, Stephanie Valencia, and Thijs Roumen. 2025. Why So Serious? Exploring Timely Humorous Comments in AAC Through AI-Powered Interfaces. In Proceedings of the 2025 CHI (CHI ’25). ACM, Article 759, 19 pages.
Justin D. Weisz, Jessica He, Michael Muller, Gabriela Hoefer, Rachel Miles, and Werner Geyer. 2024. Design Principles for Generative AI Applications. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 378, 22 pages. DOI: 10.1145/3613904.3642466
Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022. Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In Proceedings of the 2022 CHI conference on human factors in computing systems. 1–22.
Haijun Xia, Tony Wang, Aditya Gunturu, Peiling Jiang, William Duan, and Xiaoshuo Yao. 2023. CrossTalk: Intelligent substrates for language-oriented interaction in video-based communication and collaboration. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–16.
Wei Xiang, Hanfei Zhu, Suqi Lou, Xinli Chen, Zhenghua Pan, Yuping Jin, Shi Chen, and Lingyun Sun. 2024. SimUser: Generating Usability Feedback by Simulating Various Users Interacting with Mobile Applications. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 9, 17 pages. DOI: 10.1145/3613904.3642481
Shuchang Xu, Chang Chen, Zichen Liu, Xiaofu Jin, Lin-Ping Yuan, Yukang Yan, and Huamin Qu. 2024. Memory Reviver: Supporting Photo-Collection Reminiscence for People with Visual Impairment via a Proactive Chatbot. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. 1–17.
Litao Yan, Alyssa Hwang, Zhiyuan Wu, and Andrew Head. 2024. Ivie: Lightweight Anchored Explanations of Just-Generated Code. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 140, 15 pages. DOI: 10.1145/3613904.3642239
Sojeong Yun and Youn-kyung Lim. 2025. User Experience with LLM-powered Conversational Recommendation Systems: A Case of Music Recommendation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 898, 15 pages. DOI: 10.1145/3706598.3713347
Guanhua Zhang, Mohamed Adel Naguib Ahmed, Zhiming Hu, and Andreas Bulling. 2025. Summact: Uncovering user intentions through interactive behaviour summarisation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–17.
Mengxi Zhang, Huaxiao Liu, Changhao Du, Tengmei Wang, Han Li, Pei Huang, and Chunyang Chen. 2025. Distinguishing GUI Component States for Blind Users using Large Language Models. ACM Transactions on Software Engineering and Methodology (2025).
Tianhao Zhang, Fu Peiguo, Jie Liu, Yihe Zhang, and Xingmei Chen. 2024. NLDesign: A UI Design Tool for Natural Language Interfaces. In Proceedings of the ACM Turing Award Celebration Conference - China 2024 (Changsha, China) (ACM-TURC ’24). Association for Computing Machinery, New York, NY, USA, 153–158. DOI: 10.1145/3674399.3674455
Qingxiao Zheng, Minrui Chen, Pranav Sharma, Yiliu Tang, Mehul Oswal, Yiren Liu, and Yun Huang. 2025. EvAlignUX: Advancing UX Evaluation through LLM-Supported Metrics Exploration. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). ACM, Article 1051, 25 pages.
Mingyuan Zhong, Ruolin Chen, Xia Chen, James Fogarty, and Jacob OWobbrock. 2025. ScreenAudit: Detecting Screen Reader Accessibility Errors in Mobile Apps Using Large Language Models. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19.
Ting Zhou, Yanjie Zhao, Xinyi Hou, Xiaoyu Sun, Kai Chen, and Haoyu Wang. 2025. DeclarUI: Bridging Design and Development with Automated Declarative UI Code Generation. Proc. ACM Softw. Eng. 2, FSE, Article FSE011 (June 2025), 23 pages.
Publicado
10/11/2025
Como Citar
SANTOS, Simone de Oliveira; ALVES, Robson de Moraes; OLIVEIRA, Francisco Carlos de Mattos Brito; ARAUJO, Francisco Ivanilso Soares.
Investigating LLM-based tools to support Usability, Accessibility, User Experience in HCI activities: A Systematic Literature Mapping. In: BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB (WEBMEDIA), 31. , 2025, Rio de Janeiro/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 631-645.
DOI: https://doi.org/10.5753/webmedia.2025.16160.
