Impact of combinatorial optimization on reinforcement learning for stock trading in financial markets

• Guilherme Dourado Santos USP
• Karla Roberta Lima USP

Resumo

Context: In recent years, discussions have grown about ideal machine learning techniques for financial applications. Reinforcement Learning (RL) has been gaining prominence in decision-making systems, however it depends on subjective reward functions, generating the need for new methodologies. Problem: The financial market’s complexity, characterized by diverse influencing factors, poses challenges for accurate prediction. RL algorithms, relying on reward metrics, may lack objectivity in determining optimal rewards, impacting decision-making accuracy. Solution: This work proposes an approach for trading financial assets, merging RL and Combinatorial Optimization. The methodology involves a two-pronged process: a Combinatorial Optimization model determines the maximum profit achievable in a given trading period, influencing the reward function of a second RL model. IS Theory: Grounded in the Theory of Computational Learning, this work explores the application of machine learning methodologies to enhance computational decision-making. The integration of Reinforcement Learning and Combinatorial Optimization aligns with principles of learning and adaptation in computational systems. Method: The models are trained and tested on historical stock prices, particularly focusing on Petrobras’ stock, a Brazilian petroleum company. Summary of Results: Using financial market metrics, it was possible to observe that the proposed model managed to perform better than the baseline model. This achievement is influenced by the maximized profit obtained through the Combinatorial Optimization model, demonstrating the effectiveness of the proposed approach. Contributions and Impact in the IS area: The results demonstrate that combining combinatorial optimization with reinforcement learning enhances model performance. The versatility of this approach extends beyond finance, making it applicable to various domains within Information Systems.

Palavras-chave: Combinatorial Optimization, Finance, Machine learning, Reinforcement Learning