A Multidimensional Analysis of Swarm Dynamics in a Reinforcement Learning-Based Metaheuristic

• Rodrigo Cesar Lira UPE
• Mariana Macedo Northeastern University London / Northeastern University Boston
• Hugo Valadares Siqueira UTFPR
• Carmelo Bastos-Filho UPE

Resumo

Traditional assessments of metaheuristics typically focus on performance metrics when solving benchmark problems, with limited attention to the underlying mechanisms that drive collective intelligence. This performance-centric approach fails to indicate why certain metaheuristics are successful in specific scenarios and provides minimal guidance for improving their design. To better understand these inner dynamics, we conducted a multidimensional analysis of the swarm dynamics in a Reinforcement Learning-based metaheuristic that can combine search behaviors from different Swarm Intelligence metaheuristics. Using Interaction Networks, Search Trajectory Networks, Improvement Frequency, and Population Turnover metrics simultaneously, we reveal how the RL approach dynamically switches between different swarm behaviors depending on the problem. Our results demonstrate that the RL-based metaheuristic matches the performance of the most effective specialised algorithm for each benchmark function by adaptively mimicking its social interaction patterns, search trajectories, and fitness development. We show that for the F1 function, the RL approach adopts behaviors similar to GWO, while for the F2 function, it transitions to GPSO-like patterns, aligning with the expected best-performing characteristics reported in the literature. This multidimensional approach provides insights into how adaptive metaheuristics combine different behaviours at optimisation stages, suggesting a pathway toward self-adaptive metaheuristics that automatically select optimal search strategies according to the simulation scenario.