Can LLMs Solve Ordinary Differential Equations? A Comparative Analysis

• Domingos Samanjata Universidade de São Paulo (USP) / Universidade José Eduardo dos Santos (UJES) https://orcid.org/0009-0003-0261-1710
• René Vieira Santin Universidade de São Paulo (USP) https://orcid.org/0009-0007-6994-1510
• Ricardo Marcondes Marcacini Universidade de São Paulo (USP) https://orcid.org/0000-0002-2309-3487
• Solange Oliveira Rezende Universidade de São Paulo (USP) https://orcid.org/0000-0002-5233-7639

DOI: https://doi.org/10.5753/sbbd.2026.249229

Resumo

Solving Ordinary Differential Equations (ODEs) requires mathematical modeling and reasoning skills that remain challenging for large language models (LLMs). This paper evaluates five LLMs, DeepSeek-R1, Claude-Haiku-4.5, Llama-3.3-70B-Instruct, Gemini-2.5-Pro, and GPT-5.2, using ten first-order ODE application problems and a task involving analogous problem generation. We propose a multi-dimensional framework assessing mathematical modeling, Python code generation, solution accuracy, and the generation of analogous problems. Results show strong modeling performance across all models (96%–98%), demonstrating their ability to translate natural language descriptions into mathematical formulations; when generating analogous problems, scores ranged from 85% to 100%, enabling direct performance comparisons against each model’s own baseline. However, performance declines in code generation tasks, exposing limitations in multi-step reasoning and execution. Claude-Haiku-4.5 achieves the highest overall score (84%), while Llama-3.3-70B-Instruct performs best in code generation. GPT-5.2 records the lowest overall performance. These findings reveal a gap between conceptual understanding and reliable end-to-end problem solving, highlighting the importance of human oversight and hybrid symbolic-numerical approaches.

Palavras-chave: Large Language Models, Ordinary Differential Equations, Mathematical Modeling, Analogous Problem Generation, Benchmarking Evaluation

Referências

Chen, S. F. and Goodman, J. (1999). An empirical study of smoothing techniques for language modeling. Computer Speech & Language, 13(4):359–394.

Gao, L. et al. (2023). PAL: Program-Aided Language Models. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023), volume 202 of Proceedings of Machine Learning Research, pages 10764–10799. PMLR.

Glazer, E. et al. (2024). Frontiermath: A benchmark for evaluating advanced mathematical reasoning in ai. arXiv preprint arXiv:2411.04872.

Gu, J., Jiang, X., Shi, Z., Tan, H., Zhai, X., Xu, C., Li, W., Shen, Y., Ma, S., Liu, H., et al. (2024). A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594.

Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. (2021). Measuring mathematical problem solving with the math dataset. In Proceedings of the Thirty-Fifth Conference on Neural Information Processing Systems (NeurIPS 2021), Datasets and Benchmarks Track.

Huang, X., Shen, Q., Hu, Y., Gao, A., and Wang, B. (2025). Llms for mathematical modeling: Towards bridging the gap between natural and mathematical languages. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 2678–2710. Association for Computational Linguistics, Albuquerque, New Mexico.

Lee, G.-G., Latif, E., Wu, X., Liu, N., and Zhai, X. (2024). Applying large language models and chain-of-thought for automatic scoring. Computers and Education: Artificial Intelligence, 6:100213.

Nikolaiev, A., Stathopoulos, Y., and Teufel, S. (2024). Can language models rival mathematics students? evaluating mathematical reasoning through textual manipulation and human experiments. arXiv preprint arXiv:2412.11908.

Samanjata, D., Santin, R. V., Marcacini, R. M., and Rezende, S. O. (2025). Benchmarking large language models for solving ordinary differential equations. In Proceedings of the International Conference on Smart Computing and Machine Intelligence (ISCMI).

Sun, Y., Hu, S., Zhou, G., Zheng, K., Hajishirzi, H., Dziri, N., and Song, D. (2025). Omega: Can llms reason outside the box in math? evaluating exploratory, compositional, and transformative generalization. Advances in Neural Information Processing Systems, 38.

Yue, J. (2025). Benchmarking llms on advanced mathematical reasoning. Technical report, University of California, Berkeley.

Zhang, L., Wang, Y., and Lee, J. (2023). Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.

Zhao, Z. et al. (2024). Stepwise self-consistent mathematical reasoning with large language models. arXiv preprint arXiv:2402.17786.

Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. (2023). Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595–46623.