On the Impact of MDP Design for Reinforcement Learning Agents in Resource Management

• Renato Luiz de Freitas Cunha UFMG https://orcid.org/0000-0002-3196-3008
• Luiz Chaimowicz UFMG https://orcid.org/0000-0001-8156-9941

Resumo

The recent progress in Reinforcement Learning applications to Resource Management presents Markov Decision Processes (MDPs) without a deeper analysis of the impacts of design decisions on agent performance. In this paper, we compare and contrast four different MDP variations, discussing their computational requirements and impacts on agent performance by means of an empirical analysis. We conclude by showing that, in our experiments, when using Multi-Layer Perceptrons as approximation function, a compact state representation allows transfer of agents between environments, and that transferred agents have good performance and outperform specialized agents in 80% of the tested scenarios, even without retraining.

Palavras-chave: Reinforcement Learning, Resource management, Markov Decision Processes