MALiBU: Metaheuristics Approach for Online Load Balancing in MapReduce with Skewed Data Input

  • Matheus H. M. Pericini
  • Lucas G. M. Leite
  • Javam C. Machado

Resumo


MapReduce is a parallel computing model where a large dataset is split into smaller parts and executed on multiple machines. When data are not uniformly distributed, we have the so called partitioning skew, where the allocation of tasks to machines becomes unbalanced, either by the distribution function splitting the dataset unevenly or because a part of the data is more complex and requires greater computing effort. To solve this problem, we propose a function based on Simulated Annealing metaheuristic which finds a partitioning that results in a better load balancing.
Publicado
19/05/2017
PERICINI, Matheus H. M.; LEITE, Lucas G. M.; MACHADO, Javam C.. MALiBU: Metaheuristics Approach for Online Load Balancing in MapReduce with Skewed Data Input. In: SIMPÓSIO BRASILEIRO DE REDES DE COMPUTADORES E SISTEMAS DISTRIBUÍDOS (SBRC), 35. , 2017, Belém. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2017 . ISSN 2177-9384.