A Dynamic Vertical Scaling Strategy for Distributed Stream Processing Applications in Edge Computing

• Guilherme Hiago Costa dos Santos PUCRS
• Carlos Henrique Kayser PUCRS
• Tiago Ferreto PUCRS

DOI: https://doi.org/10.5753/sscad.2026.34985

Resumo

Distributed Stream Processing applications at the edge must reconcile low latency and high throughput with limited and heterogeneous resources. This paper presents a dynamic vertical scaling strategy based on Proximal Policy Optimization, formulated as a Partially Observable Markov Decision Process. The policy jointly adjusts task allocations and prioritizes compliance with a service-level objective for the 95th-percentile end-to-end latency. In simulation-based experiments with two application profiles, two workloads, and ten paired placements per combination, the selected policies preserved throughput, achieved mean SLO violation rates ranging from 0.03% to 0.30%, with reductions of up to 8.55 percentage points compared with the baseline solution, and also used less CPU in three of four combinations. The comparison covers complete controller configurations with different decision frequencies.

Referências

Arkian, H., Pierre, G., Tordsson, J., and Elmroth, E. (2021). Model-based stream processing auto-scaling in geo-distributed environments. In 2021 International Conference on Computer Communications and Networks (ICCCN), pages 1–10. IEEE.

Belkhiria, M. M. and Tedeschi, C. (2019). Design and evaluation of decentralized scaling mechanisms for stream processing. In 2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), pages 247–254. IEEE.

Bodik, P., Hong, W., Guestrin, C., Madden, S., Paskin, M., and Thibaux, R. (2004). Intel lab data. [link]. Data collected from 54 sensors deployed in the Intel Berkeley Research lab.

Ching, C.-W., Chen, X., Kim, C., Wang, T., Chen, D., Da Silva, D., and Hu, L. (2025). AgileDART: An agile and scalable edge stream processing engine. IEEE Transactions on Mobile Computing, 24(5):4510–4528.

Kang, P., Lama, P., and Khan, S. U. (2021). SLO-aware virtual rebalancing for edge stream processing. In 2021 IEEE International Conference on Cloud Engineering (IC2E), pages 126–135. IEEE.

Nardelli, M., Russo Russo, G., Cardellini, V., and Lo Presti, F. (2018). A multi-level elasticity framework for distributed data stream processing. In European Conference on Parallel Processing, pages 53–64. Springer.

Peng, Q., Xia, Y., Wang, Y., Wu, C., Luo, X., and Lee, J. (2019). Joint operator scaling and placement for distributed stream processing applications in edge computing. In Service-Oriented Computing: 17th International Conference, ICSOC 2019, pages 461–476. Springer.

Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.

Shi, W., Cao, J., Zhang, Q., Li, Y., and Xu, L. (2016). Edge computing: Vision and challenges. IEEE Internet of Things Journal, 3(5):637–646.

Shi, W. and Dustdar, S. (2016). The promise of edge computing. Computer, 49(5):78–81.

Shukla, A., Chaturvedi, S., and Simmhan, Y. (2017). RIoTBench: An iot benchmark for distributed stream processing systems. Concurrency and Computation: Practice and Experience, 29(21):e4257.

Souza, P. S., Ferreto, T., and Calheiros, R. N. (2023). EdgeSimPy: Python-based modeling and simulation of edge computing resource management policies. Future Generation Computer Systems, 148:446–459.

Xu, J. and Palanisamy, B. (2021). Model-based reinforcement learning for elastic stream processing in edge computing. In 2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC), pages 292–301. IEEE.