The Impact of Automated Reinforcement Learning Fine-Tuning on Advanced Prompt Engineering for Code-Based Reasoning

• Guilherme Louro de Salignac e Souza UFAM
• Marco Cristo UFAM
• Eulanda Miranda dos Santos UFAM

DOI: https://doi.org/10.5753/eniac.2026.31154

Resumo

Prompt Engineering (PE) techniques such as Program of Thoughts (PoT) and Structured Chain-of-Thought (SCoT) were originally proposed to induce structured reasoning in Large Language Models (LLMs). However, the emergence of Large Reasoning Models (LRMs), which internalize reasoning through Reinforcement Learning, raises questions about whether such techniques remain useful. This study compares Zero-shot, Chain-of-Thought (CoT), PoT, and SCoT on an LLM, LLaMA-3.1-8B-Instruct, and its corresponding LRM, DeepSeek-R1-Distill-LLaMA-8B, using the GSM8K dataset. DeepSeek significantly outperformed LLaMA across all prompt strategies, with gains ranging from +26.7 to +43.7 p.p. Within LLaMA, structured prompting mainly induced reasoning, with SCoT achieving the best performance. Within DeepSeek, CoT did not improve over Zero-shot, while PoT and SCoT achieved statistically similar accuracy despite the higher inferential cost of SCoT. These results suggest that PE remains relevant for LRMs, but its role shifts from eliciting reasoning to modulating its structure and cost.

Referências

Chen, W. et al. (2023). Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research.

Cobbe, K. et al. (2021). Training verifiers to solve math word problems.

Guo, D. et al. (2025). Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature, 645(8081):633–638.

Jiang, B. et al. (2024). A peek into token bias: Large language models are not yet genuine reasoners. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4722–4756.

Li, J. et al. (2025). Structured chain-of-thought prompting for code generation. ACM Transactions on Software Engineering and Methodology, 34(2):1–23.

Marjanovic, S. V. et al. (2026). Deepseek-r1 thoughtology: Let’s think about LLM reasoning. Transactions on Machine Learning Research.

Mondorf, P. and Plank, B. (2024). Beyond accuracy: Evaluating the reasoning behavior of large language models – a survey.

Muennighoff, N. et al. (2025). s1: Simple test-time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20286–20332.

OpenAI (2024). Openai o1. [link]. Accessed on: 2025-01-31.

Sahoo, P. et al. A systematic survey of prompt engineering in large language models: Techniques and applications.

Shi, F. et al. (2023). Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pages 31210–31227. PMLR.

Wei, J. et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.

Xu, F. et al. (2025). Toward large reasoning models: A survey of reinforced reasoning with large language models. Patterns, 6(10):101370.

Yang, D. et al. Code to think, think to code: A survey on code-enhanced reasoning and reasoning-driven code intelligence in llms.