Analyzing C++ Stream Parallelism in Shared-Memory when Porting to Flink and Storm

  • Renato B. Hoffmann PUCRS
  • Leonardo G. Faé PUCRS
  • Isabel H. Manssour PUCRS
  • Dalvan Griebler PUCRS

Resumo


Stream processing plays a crucial role in various information-oriented digital systems. Two popular frameworks for real-time data processing, Flink and Storm, provide solutions for effective parallel stream processing in Java. An option to leverage Java's mature ecosystem for distributed stream processing involves porting legacy C++ applications to Java. However, this raises considerations on the adequacy of the equivalent Java mechanisms and potential degradation in throughput. Therefore, our objective is to evaluate programmability and performance when converting stream processing applications from C++ to Java while also exploring the parallelization capabilities offered by Flink and Storm. Furthermore, we aim to assess the throughput of Flink and Storm on shared-memory manycore machines, a hardware architecture commonly found in cloud environments. To achieve this, we conduct experiments involving four different stream processing applications. We highlight challenges encountered when porting C++ to Java and working with Flink and Storm. Furthermore, we discuss throughput, latency, CPU, and memory usage results.
Palavras-chave: Flin, Storm, Parallelism
Publicado
17/10/2023
HOFFMANN, Renato B.; FAÉ, Leonardo G.; MANSSOUR, Isabel H.; GRIEBLER, Dalvan. Analyzing C++ Stream Parallelism in Shared-Memory when Porting to Flink and Storm. In: WORKSHOP ON APPLICATIONS FOR MULTI-CORE ARCHITECTURES - INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE AND HIGH PERFORMANCE COMPUTING (SBAC-PAD), 35. , 2023, Porto Alegre/RS. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2023 . p. 46-53.