Comparative Analysis of LLMs in Database Index Optimization: A Study with GPT-4, Claude 3.7 Sonnet, and Gemini 2.0 Flash

• Daniel Costa IFPB
• Danyllo Albuquerque IFPB
• Bruno Moreno IFPB

DOI: https://doi.org/10.5753/encompif.2025.9369

Abstract

This study presents a comparative analysis of three Large Language Models (LLMs) — GPT-4, Claude 3.7 Sonnet, and Gemini 2.0 Flash — focusing on their ability to suggest indexes for performance optimization in relational databases. Using a PostgreSQL system hosted on Supabase, indexing suggestions from each model were applied, and execution times of a standardized set of SQL queries were evaluated. The results showed that, although all models provided valid recommendations, only GPT-4 exhibited minimal performance impact, with the lowest average execution time increase (+0.097ms). Claude 3.7 Sonnet showed intermediate performance (+0.102ms), suggesting additional elements such as constraints, while Gemini 2.0 Flash resulted in the highest overhead (+0.149ms), indicating lower practical effectiveness. This research contributes to understanding LLMs as support tools in data engineering, providing empirical evidence of their applicability and limitations in automated database optimization scenarios.

References

Date, C. J. (2004). Introdução a sistemas de bancos de dados. Elsevier Brasil.

de Menezes, I. V. C. B., de Oliveira Barbosa, L. S., and Garzon, N. A. (2023). Aplicações de inteligência artificial na otimização de banco de dados. RECIMA21-Revista Científica Multidisciplinar-ISSN 2675-6218, 4(12):e4124516–e4124516.

Heuser, C. A. (2009). Projeto de banco de dados, volume 6. Bookman Porto Alegre.

Hong, Z., Yuan, Z., Zhang, Q., Chen, H., Dong, J., Huang, F., and Huang, X. (2024). Next-generation database interfaces: A survey of llm-based text-to-sql. arXiv preprint arXiv:2406.08426.

Jingru Gan, Peichen Zhong, Y. D. Y. Z. C. D. H. W. C. P. G. K. A. P. D. S.-K. W. W. (2025). Large language models are innate crystal structure generators. arXiv:2502.20933.

Pereira, P. C. (2021). Introdução a banco de dados. São Paulo: Editora Senac.

Sriram Dharwada, Himanshu Devrani, J. H. H. D. (2025). Query rewriting via llms. arXiv:2502.12918.

Tan, J., Zhao, K., Li, R., Yu, J. X., Piao, C., Cheng, H., Meng, H., Zhao, D., and Rong, Y. (2025). Can large language models be query optimizer for relational databases? arXiv preprint arXiv:2502.05562.

Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., and Zhang, Y. (2024). A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211.

Zhou, X., Sun, Z., and Li, G. (2024). Db-gpt: Large language model meets database. Data Science and Engineering, 9(1):102–111.