Telemetria Adaptativa Usando Aprendizado por Reforço Profundo em Redes Definidas por Software

• Debora H. Job UNIRIO
• Sidney C. de Lucena UNIRIO
• Pedro Nuno Moura UNIRIO

DOI: https://doi.org/10.5753/sbrc.2023.441

Abstract

The use of software-defined networks leveraged the adoption of telemetry techniques for fine-grained monitoring. However, its indiscriminate adoption generates additional costs. It may degrade network performance, increasing the amount of the stored data to be processed, making its benefits unfeasible. The adoption of adaptive telemetry then emerges as a way to circumvent this problem. This work proposes a deep reinforcement learning engine to provide a data plane adaptive telemetry that monitors congestion. As a proof-of-concept, we developed an environment on a version of the ONOS platform enabled with the In-Band Network Telemetry (INT) engine and P4 switches. Experiments with varying traffic profiles and hyperparameters confirm the proposal’s benefits and explore its limitations.

References

Arulkumaran, K., Deisenroth, M. P., Brundage, M., and Bharath, A. A. (2017). Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6):26-38.

Bosshart, P., Daly, D., Gibb, G., Izzard, M., McKeown, N., Rexford, J., Schlesinger, C., Talayco, D., Vahdat, A., Varghese, G., and Walker, D. (2014). P4: Programming protocol-independent packet processors. SIGCOMM Comput. Commun. Rev., 44(3):87-95.

Graesser, L. and Keng, W. L. (2020). Foundations of Deep Reinforcement Learning Theory and Practice in Python. Addison Wesley Data & Analytics Series.

Hyun, J., Van Tu, N., and Hong, J. W.-K. (2018). Towards knowledge-defined networking using in-band network telemetry. In NOMS 2018 2018 IEEE/IFIP Network Operations and Management Symposium, pages 1-7.

Jouet, S. and Pezaros, D. P. (2017). Bpfabric: Data plane programmability for software defined networks. In 2017 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), pages 38-48.

Krohn, J. and Beyleveld, G. ang Bassens, A. (2020). Deep Learning Illustrated. A Visual, Interactive Guide to Artificial Intelligence. Addison Wesley Data & Analytics Series.

Mestres, A., Rodriguez-Natal, A., Carner, J., Barlet-Ros, P., Alarcó n, E., Solé, M., Muntés-Mulero, V., Meyer, D., Barkai, S., Hibbett, M. J., Estrada, G., Ma'ruf, K., Coras, F., Ermagan, V., Latapie, H., Cassar, C., Evans, J., Maino, F., Walrand, J., and Cabellos, A. (2017). Knowledge-defined networking. ACM SIGCOMM Computer Communication Review, 47(3):2-10.

Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540):529-533.

Poole, D. L. and Mackworth, A. K. (2017). Artificial Intelligence: Foundations of Computational Agents. Cambridge University Press, USA, 2nd edition.

Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction. The MIT Press, second edition.

Tangari, G., Tuncer, D., Charalambides, M., Qi, Y., and Pavlou, G. (2018). Self-adaptive decentralized monitoring in software-defined networks. IEEE Transactions on Network and Service Management, 15(4):1277-1291.

Tu, N. V., Hyun, J., Kim, G. Y., Yoo, J.-H., and Hong, J. W.-K. (2018). Intcollector: A high-performance collector for in-band network telemetry. In 2018 14th International Conference on Network and Service Management (CNSM), pages 10-18.

Van Tu, N., Hyun, J., and Hong, J. W.-K. (2017). Towards onos-based sdn monitoring using in-band network telemetry. In 2017 19th Asia-Pacific Network Operations and Management Symposium (APNOMS), pages 76-81.

Xie, J., Yu, F. R., Huang, T., Xie, R., Liu, J., Wang, C., and Liu, Y. (2019). A survey of machine learning techniques applied to software defined networking (sdn): Research issues and challenges. IEEE Communications Surveys & Tutorials, 21(1):393-430.

Yao, H., Mai, T., Xu, X., Zhang, P., Li, M., and Liu, Y. (2018). Networkai: An intelligent network architecture for self-learning control strategies in software defined networks. IEEE Internet of Things Journal, 5(6):4319-4327.

Yu, M. (2019). Network telemetry: Towards a top-down approach. SIGCOMM Comput. Commun. Rev., 49(1):11-17.