On the Impact of MDP Design for Reinforcement Learning Agents in Resource Management

Resumo


The recent progress in Reinforcement Learning applications to Resource Management presents Markov Decision Processes (MDPs) without a deeper analysis of the impacts of design decisions on agent performance. In this paper, we compare and contrast four different MDP variations, discussing their computational requirements and impacts on agent performance by means of an empirical analysis. We conclude by showing that, in our experiments, when using Multi-Layer Perceptrons as approximation function, a compact state representation allows transfer of agents between environments, and that transferred agents have good performance and outperform specialized agents in 80% of the tested scenarios, even without retraining.
Palavras-chave: Reinforcement Learning, Resource management, Markov Decision Processes
Publicado
29/11/2021
CUNHA, Renato Luiz de Freitas; CHAIMOWICZ, Luiz. On the Impact of MDP Design for Reinforcement Learning Agents in Resource Management. In: BRAZILIAN CONFERENCE ON INTELLIGENT SYSTEMS (BRACIS), 10. , 2021, Online. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2021 . ISSN 2643-6264.