Evaluating ChatGPT to support data visualization design
Abstract
Large Language Models (LLMs) like ChatGPT offer new avenues for diverse tasks, including complex activities such as data visualization design. While existing studies explore LLMs for specific design activities, particularly product generation and ideation, a comprehensive investigation into their support for the entire data visualization design process, especially for non-experts, remains largely unexplored. This work addresses this gap by investigating LLMs' capability to assist beginners in applying design methods and tools throughout the data visualization design process. Our research was guided by the question: “How can LLMs support the visualization design process?” A preliminary study explored prompt strategies for generating data visualization recommendations and established criteria for evaluating model response quality. Methodologically, we analysed design techniques, created prompts for non-specialist designers, and evaluated the process of building design guides with experts, defining a usage context and evaluation criteria for assessing user interaction and perception. Our findings indicate that ChatGPT can support both abstract and tangible design activities at varying levels. However, it is crucial that users possess sufficient domain knowledge to critically evaluate and refine the model’s outputs, ensuring accuracy and appropriateness in data visualization design.
References
Bischof, N., Comi, A., e Eppler, M. J. (2011). Knowledge visualization in qualitative methods – or how can i see what i say? In 2011 15th International Conference on Information Visualisation, page 371–376.
Carvalho, F. A. N. d. (2022). Compreende: um framework para a seleção de ferramentas no desenvolvimento de projetos de produtos digitais. Repositório da Universidade Federal do Ceará. Accepted: 2022-09-12T15:16:36Z.
Chen, N., Zhang, Y., Xu, J., Ren, K., e Yang, Y. (2025). Viseval: A benchmark for data visualization in the era of large language models. IEEE Transactions on Visualization and Computer Graphics, 31(1):1301–1311.
Grice, Kimball, J. P., Morgan, J. L., e Cole, P. (1975). Syntax and semantics. Academic press, Harcourt Brace Jovanovich, New York San Francisco London.
Kim, N. W., Ahn, Y., Myers, G., e Bach, B. (2025). How good is chatgpt in giving advice on your visualization design? ACM Trans. Comput.-Hum. Interact.
Kirk, A. A. (2016). Data Visualisation: A Handbook for Data Driven Design. Sage Publications, Los Angeles London New Delhi Singapore Washington DC Melbourne, 1ª edição edition.
Knafic, C. N. (2015). Storytelling with data: a data visualization guide for business professionals. Wiley, Hoboken, New Jersey.
Kumar, V. (2013). 101 design methods: a structured approach for driving innovation in your organization. Wiley, Hoboken, N.J.
McKenna, S., Mazur, D., Agutter, J., e Meyer, M. (2014). Design activity framework for visualization design. IEEE Transactions on Visualization and Computer Graphics, 20(12):2191–2200.
Munzner, T. (2014). Visualization Analysis and Design. AK Peters Visualization Series. CRC Press.
Nguyen, T. H., Waizenegger, L., e Techatassanasoontorn, A. A. (2022). “don’t neglect the user!”–identifying types of human-chatbot interactions and their associated characteristics. Information Systems Frontiers, 24(3):797–838.
Parsons, P. (2022). Understanding data visualization design practice. IEEE Transactions on Visualization and Computer Graphics, 28(1):665–675.
Rodrigues Catalano, J. V. e Rossi Lorenzi, B. (2023). Sem referências: o ChatGPT sob a perspectiva latouriana do duplo clique. Revista Faz Ciência, 25(41).
Stickdorn, M., Hormess, M., Lawrence, A., e Schneider, J., editors (2021). This is service design doing: applying service design thinking in the real world ; a practitioners’ handbook. O’Reilly Media, Sebastopol, CA, 10. nachdr edition.
Van Someren, M., Barnard, Y. F., e Sandberg, J. (1994). The think aloud method: a practical approach to modelling cognitive. London: AcademicPress, 11(6).
