Towards Small Language Models for Text-to-SQL

• Paulo H. C. Silva UFV
• Letícia O. Silva UFV
• Fabrício A. Silva UFV

Resumo

The task of text-to-SQL, which involves translating natural language queries into structured SQL queries, is an industry demand that enables non-technical users to interact with relational databases. However, current state-of-the-art approaches primarily rely on Large Language Models (LLMs), which pose limitations such as high computational resource consumption and, when closed-source, reliability concerns. This research explores how Small Language Models (SLMs), specifically open-source models with up to 3 billion parameters, can be leveraged to enhance the text-to-SQL process through supervised fine-tuning, with a focus on incorporating query normalization and schema-linking to improve model performance. The findings highlight that SLMs can establish themselves as feasible alternatives to democratize the adoption of generative AI in business contexts, and schema-linking proves to be even more critical in the pipeline as the number of model parameters decreases, allowing improvements in execution accuracy (EX) ranging from 6.9% to 46.9%. This work was motivated by a business demand from a small company with limited computational resources. We have deployed an application powered by our fine-tuned SLM using a single NVIDIA A10 GPU with 24 GB of VRAM, validated through a case study using a real-world business database, achieving up to 97% EX for easy queries and 64.5% in general, without the need for domain adaptation, demonstrating the practical effectiveness and generalizability of our approach.