Evaluating ChatGPT to Support Data Visualization Design

• George M. Oliveira PUC-Rio
• Simone D. J. Barbosa PUC-Rio

DOI: https://doi.org/10.5753/ihc_estendido.2024.242031

Resumo

Grandes modelos de linguagem (LLMs) podem ajudar a recuperar as informações para responder perguntas, construir imagens e áudios, e auxiliar em atividades complexas como o design de visualização de dados. Este último requer conhecimentos específicos que podem estar disponíveis na internet e utilizados para treinar LLMs. Este trabalho investiga a capacidade do ChatGPT para auxiliar no design de visualização de dados. Conduzimos uma avaliação do modelo com base em métricas e planejamos expandi-la para entender a visão dos usuários criadores de visualização, sejam ou não especialistas.

Referências

Alexandra Farazouli, Teresa Cerratto-Pargman, K. B.-L. and McGrath, C. (2024). Hello GPT! Goodbye home examination? An exploratory study of AI chatbots impact on university teachers’ assessment practices. Assessment & Evaluation in Higher Education, 49(3):363–375.

Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., Ye, W., Zhang, Y., Chang, Y., Yu, P. S., Yang, Q., and Xie, X. (2024). A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol., 15(3).

Chenfu, S., Shoji, Y., Yamamoto, T., Tanaka, K., and Dürst, M. J. (2024). Generating experiential descriptions and estimating evidence using generative language model and user products reviews. In 2024 IEEE International Conference on Big Data and Smart Computing (BigComp), pages 254–261.

Kim, N. W., Myers, G., and Bach, B. (2024). How good is ChatGPT in giving advice on your visualization design? arXiv:2310.09617.

Maddigan, P. and Susnjak, T. (2023). Chat2VIS: Generating data visualizations via natural language using ChatGPT, codex and GPT-3 large language models. IEEE Access, 11:45181–45193.

Mulia, A. P., Piri, P. R., and Tho, C. (2023). Usability analysis of text generation by ChatGPT OpenAI using system usability scale method. Procedia Computer Science, 227:381–388. 8th International Conference on Computer Science and Computational Intelligence (ICCSCI 2023).

Narechania, A., Srinivasan, A., and Stasko, J. (2021). NL4DV: A toolkit for generating analytic specifications for data visualization from natural language queries. IEEE Transactions on Visualization & Computer Graphics, 27(02):369–379.

Rodrigues Catalano, J. V. and Rossi Lorenzi, B. (2023). Sem referências: o chatgpt sob a perspectiva latouriana e a armadilha do duplo clique. Revista Faz Ciência, 25(41).

Skjuve, M., Følstad, A., and Brandtzaeg, P. B. (2023). The user experience of chatgpt: Findings from a questionnaire study of early users. In Proceedings of the 5th International Conference on Conversational User Interfaces, CUI ’23, page 1–10, New York, NY, USA. Association for Computing Machinery.

Sun, Y., Leigh, J., Johnson, A., and Lee, S. (2010). Articulate: A semi-automated model for translating natural language queries into meaningful visualizations. In Taylor, R., Boulanger, P., Krüger, A., and Olivier, P., editors, Smart Graphics, pages 184–195, Berlin, Heidelberg. Springer Berlin Heidelberg.

Wei, Q., Yao, Z., Cui, Y., Wei, B., Jin, Z., and Xu, X. (2024). Evaluation of chatgpt-generated medical responses: A systematic review and meta-analysis. Journal of Biomedical Informatics, 151:104620.

Zhu, N., Zhang, N., Shao, Q., Cheng, K., and Wu, H. (2024). OpenAI’s GPT-4o in surgical oncology: Revolutionary advances in generative artificial intelligence. European Journal of Cancer, 206:114132.