Can LLMs Solve Ordinary Differential Equations? A Comparative Analysis - Samanjata, Domingos; Santin, René Vieira; Marcacini, Ricardo Marcondes; Rezende, Solange Oliveira (Anais do Simpósio Brasileiro de Banco de Dados (SBBD); 2026: Anais do XLI Simpósio Brasileiro de Bancos de Dados, 2026)
Solving Ordinary Differential Equations (ODEs) requires mathematical modeling and reasoning skills that remain challenging for large language models (LLMs). This paper evaluates five LLMs, DeepSeek-R1, Claude-Haiku-4.5, Llama-3.3-70B-Instruct, Gemini-2.5-Pro, and GPT-5.2, using ten first-order ODE application problems and a task involving analogous problem generation. We propose a multi-dimensional framework assessing mathematical modeling, Python code generation, solution accuracy, and the generation of analogous problems. Results show strong modeling performance across all models (96%–98%), demonstrating their ability to translate natural language descriptions into mathematical formulations; when generating analogous problems, scores ranged from 85% to 100%, enabling direct performance comparisons against each model’s own baseline. However, performance declines in code generation tasks, exposing limitations in multi-step reasoning and execution. Claude-Haiku-4.5 achieves the highest overall score (84%), while Llama-3.3-70B-Instruct performs best in code generation. GPT-5.2 records the lowest overall performance. These findings reveal a gap between conceptual understanding and reliable end-to-end problem solving, highlighting the importance of human oversight and hybrid symbolic-numerical approaches.