Discovering Subgoals from Trajectories Using Empowerment - Thomasini, Luiz A.; Souza, Arturo de; Ramos, Gabriel de Oliveira (Anais da Escola Regional de Aprendizado de Máquina e Inteligência Artificial da Região Sul (ERAMIA-RS), 2025)
Reinforcement learning algorithms struggle in complex environments with sparse rewards due to inefficient exploration. Identifying subgoals to guide exploration and structure learning is a promise and a challenge. This work presents a novel method to discover subgoals from trajectories by leveraging empowerment, an information-theoretic measure of an agent’s potential to influence its environment. The main hypothesis is that states with high empowerment correspond to strategic locations, such as bottlenecks, which serve as subgoals. We evaluate our method in grid-world navigation tasks, demonstrating that it successfully identifies important states without requiring reward signals or successful trajectories, and improves agent performance.