Log-Driven Autonomic Auto-Scaling with LSTM Forecasting: An Industrial Case in On-Premises Containerized Systems

• Rafael Xavier CEFET-MG
• Rafael Durelli UFLA
• Bruno Cafeo UNICAMP
• Elder Cirilo UFSJ

DOI: https://doi.org/10.5753/sbes.2025.11133

Abstract

This work presents an autonomic auto-scaling solution for a containerized system deployed on on-premises infrastructure. The solution addresses the lack of native autoscaling — the automatic adjustment of computing resources to match workload demand — in private environments by implementing a self-managing architecture based on the MAPE-K (Monitor-Analyze-Plan-Execute) control loop, an established methodology where the system continuously monitors itself, analyzes changes, plans reactions, and executes them using existing knowledge. It incorporates a Long Short-Term Memory (LSTM) neural network to proactively forecast workload spikes, enabling informed scaling decisions before performance degradation occurs. A key innovation is the non-intrusive, log-driven approach, where system logs serve as the primary knowledge source for analysis and scaling decisions. Preliminary evaluation using a dataset collected from a production deployment at a Brazilian public institution reveals improved system responsiveness and resource utilization during peak demand. These results, derived from a real-world, industrial-scale scenario, demonstrate the practical applicability of the proposed solution and indicate that the LSTM-driven auto-scaler can maintain quality of service under variable workloads while optimizing resource usage.

Keywords: Auto-scaling, Autonomic Computing, Time Series Forecasting, On-Premises Containerized Systems

References

Saleha Alharthi, Afra Alshamsi, Anoud Alseiari, and Abdulmalik Alwarafy. 2024. Auto-Scaling Techniques in Cloud Computing: Issues and Research Directions. Sensors 24, 17 (2024), 5551.

Michael Armbrust, Armando Fox, Rean Griffith, Anthony D. Joseph, Randy Katz, Andy Konwinski, Gunho Lee, David Patterson, Ariel Rabkin, Ion Stoica, and Matei Zaharia. 2010. A View of Cloud Computing. Commun. ACM 53, 4 (2010), 50–58.

Dariusz Rafal Augustyn. 2017. Improvements of the Reactive Auto Scaling Method for Cloud Platform. In Computer Networks, Piotr Gaj, Andrzej Kwiecień, and Michał Sawicki (Eds.). Springer International Publishing, Cham, 422–431.

George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. 2015. Time series analysis: forecasting and control. John Wiley & Sons.

Javad Dogani, Reza Namvar, and Farshad Khunjush. 2023. Auto-scaling techniques in container-based cloud and edge/fog computing: Taxonomy and survey. Computer Communications 209 (2023), 120–150.

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (1997), 1735–1780.

Rob J Hyndman and Anne B Koehler. 2006. Another look at measures of forecast accuracy. International journal of forecasting 22, 4 (2006), 679–688.

Rob J Hyndman, Anne B Koehler, Ralph D Snyder, and Simone Grose. 2002. A state space framework for automatic forecasting using exponential smoothing methods. International Journal of forecasting 18, 3 (2002), 439–454.

IBM Corporation. 2006. An Architectural Blueprint for Autonomic Computing (4 ed.). Technical Report. IBM Corporation. [link]

Kiran Jewargi. 2023. Public Cloud to Cloud Repatriation Trend. Scholars Journal of Engineering and Technology 1 (2023), 1–3.

Natalya Yezhkova. 2024. Assessing the Scale of Workload Repatriation: Insights from IDC’s Server and Storage Workloads Surveys, 1H23 and 2H23. Technical Report US50903124. International Data Corporation (IDC). [link] IDC Survey Report.

Zhiqiang Zhou, Chaoli Zhang, Lingna Ma, Jing Gu, Huajie Qian, Qingsong Wen, Liang Sun, Peng Li, and Zhimin Tang. 2023. AHPA: adaptive horizontal pod autoscaling systems on alibaba cloud container service for kubernetes. In Proceedings of the AAAI conference on artificial intelligence, Vol. 37. 15621–15629.