Telemetria Adaptativa Usando Aprendizado por Reforço Profundo em Redes Definidas por Software
Resumo
O uso de redes definidas por software alavancou a adoção de técnicas de telemetria para um monitoramento de alta granularidade. Sua adoção indiscriminada, contudo, gera custos adicionais que podem degradar o desempenho da rede, provocar um volume exagerado de dados a serem armazenados e processados e, assim, inviabilizar seus benefícios. A adoção de telemetria adaptativa surge então como uma forma de contornar esse problema. Este trabalho propõe o uso de aprendizado por reforço profundo para prover uma telemetria adaptativa do plano de dados que monitore congestionamentos. Um ambiente para prova de conceito foi desenvolvido sobre uma versão da plataforma ONOS habilitada com o mecanismo In-Band Network Telemetry (INT) e switches P4. Experimentos variando perfis de tráfego e hiperparâmetros do mecanismo de aprendizado confirmam os benefícios da proposta e exploram suas limitações.
Referências
Arulkumaran, K., Deisenroth, M. P., Brundage, M., and Bharath, A. A. (2017). Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6):26-38.
Bosshart, P., Daly, D., Gibb, G., Izzard, M., McKeown, N., Rexford, J., Schlesinger, C., Talayco, D., Vahdat, A., Varghese, G., and Walker, D. (2014). P4: Programming protocol-independent packet processors. SIGCOMM Comput. Commun. Rev., 44(3):87-95.
Graesser, L. and Keng, W. L. (2020). Foundations of Deep Reinforcement Learning Theory and Practice in Python. Addison Wesley Data & Analytics Series.
Hyun, J., Van Tu, N., and Hong, J. W.-K. (2018). Towards knowledge-defined networking using in-band network telemetry. In NOMS 2018 2018 IEEE/IFIP Network Operations and Management Symposium, pages 1-7.
Jouet, S. and Pezaros, D. P. (2017). Bpfabric: Data plane programmability for software defined networks. In 2017 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), pages 38-48.
Krohn, J. and Beyleveld, G. ang Bassens, A. (2020). Deep Learning Illustrated. A Visual, Interactive Guide to Artificial Intelligence. Addison Wesley Data & Analytics Series.
Mestres, A., Rodriguez-Natal, A., Carner, J., Barlet-Ros, P., Alarcó n, E., Solé, M., Muntés-Mulero, V., Meyer, D., Barkai, S., Hibbett, M. J., Estrada, G., Ma'ruf, K., Coras, F., Ermagan, V., Latapie, H., Cassar, C., Evans, J., Maino, F., Walrand, J., and Cabellos, A. (2017). Knowledge-defined networking. ACM SIGCOMM Computer Communication Review, 47(3):2-10.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540):529-533.
Poole, D. L. and Mackworth, A. K. (2017). Artificial Intelligence: Foundations of Computational Agents. Cambridge University Press, USA, 2nd edition.
Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction. The MIT Press, second edition.
Tangari, G., Tuncer, D., Charalambides, M., Qi, Y., and Pavlou, G. (2018). Self-adaptive decentralized monitoring in software-defined networks. IEEE Transactions on Network and Service Management, 15(4):1277-1291.
Tu, N. V., Hyun, J., Kim, G. Y., Yoo, J.-H., and Hong, J. W.-K. (2018). Intcollector: A high-performance collector for in-band network telemetry. In 2018 14th International Conference on Network and Service Management (CNSM), pages 10-18.
Van Tu, N., Hyun, J., and Hong, J. W.-K. (2017). Towards onos-based sdn monitoring using in-band network telemetry. In 2017 19th Asia-Pacific Network Operations and Management Symposium (APNOMS), pages 76-81.
Xie, J., Yu, F. R., Huang, T., Xie, R., Liu, J., Wang, C., and Liu, Y. (2019). A survey of machine learning techniques applied to software defined networking (sdn): Research issues and challenges. IEEE Communications Surveys & Tutorials, 21(1):393-430.
Yao, H., Mai, T., Xu, X., Zhang, P., Li, M., and Liu, Y. (2018). Networkai: An intelligent network architecture for self-learning control strategies in software defined networks. IEEE Internet of Things Journal, 5(6):4319-4327.
Yu, M. (2019). Network telemetry: Towards a top-down approach. SIGCOMM Comput. Commun. Rev., 49(1):11-17.
Bosshart, P., Daly, D., Gibb, G., Izzard, M., McKeown, N., Rexford, J., Schlesinger, C., Talayco, D., Vahdat, A., Varghese, G., and Walker, D. (2014). P4: Programming protocol-independent packet processors. SIGCOMM Comput. Commun. Rev., 44(3):87-95.
Graesser, L. and Keng, W. L. (2020). Foundations of Deep Reinforcement Learning Theory and Practice in Python. Addison Wesley Data & Analytics Series.
Hyun, J., Van Tu, N., and Hong, J. W.-K. (2018). Towards knowledge-defined networking using in-band network telemetry. In NOMS 2018 2018 IEEE/IFIP Network Operations and Management Symposium, pages 1-7.
Jouet, S. and Pezaros, D. P. (2017). Bpfabric: Data plane programmability for software defined networks. In 2017 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), pages 38-48.
Krohn, J. and Beyleveld, G. ang Bassens, A. (2020). Deep Learning Illustrated. A Visual, Interactive Guide to Artificial Intelligence. Addison Wesley Data & Analytics Series.
Mestres, A., Rodriguez-Natal, A., Carner, J., Barlet-Ros, P., Alarcó n, E., Solé, M., Muntés-Mulero, V., Meyer, D., Barkai, S., Hibbett, M. J., Estrada, G., Ma'ruf, K., Coras, F., Ermagan, V., Latapie, H., Cassar, C., Evans, J., Maino, F., Walrand, J., and Cabellos, A. (2017). Knowledge-defined networking. ACM SIGCOMM Computer Communication Review, 47(3):2-10.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540):529-533.
Poole, D. L. and Mackworth, A. K. (2017). Artificial Intelligence: Foundations of Computational Agents. Cambridge University Press, USA, 2nd edition.
Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction. The MIT Press, second edition.
Tangari, G., Tuncer, D., Charalambides, M., Qi, Y., and Pavlou, G. (2018). Self-adaptive decentralized monitoring in software-defined networks. IEEE Transactions on Network and Service Management, 15(4):1277-1291.
Tu, N. V., Hyun, J., Kim, G. Y., Yoo, J.-H., and Hong, J. W.-K. (2018). Intcollector: A high-performance collector for in-band network telemetry. In 2018 14th International Conference on Network and Service Management (CNSM), pages 10-18.
Van Tu, N., Hyun, J., and Hong, J. W.-K. (2017). Towards onos-based sdn monitoring using in-band network telemetry. In 2017 19th Asia-Pacific Network Operations and Management Symposium (APNOMS), pages 76-81.
Xie, J., Yu, F. R., Huang, T., Xie, R., Liu, J., Wang, C., and Liu, Y. (2019). A survey of machine learning techniques applied to software defined networking (sdn): Research issues and challenges. IEEE Communications Surveys & Tutorials, 21(1):393-430.
Yao, H., Mai, T., Xu, X., Zhang, P., Li, M., and Liu, Y. (2018). Networkai: An intelligent network architecture for self-learning control strategies in software defined networks. IEEE Internet of Things Journal, 5(6):4319-4327.
Yu, M. (2019). Network telemetry: Towards a top-down approach. SIGCOMM Comput. Commun. Rev., 49(1):11-17.
Publicado
22/05/2023
Como Citar
JOB, Debora H.; LUCENA, Sidney C. de; MOURA, Pedro Nuno.
Telemetria Adaptativa Usando Aprendizado por Reforço Profundo em Redes Definidas por Software. In: SIMPÓSIO BRASILEIRO DE REDES DE COMPUTADORES E SISTEMAS DISTRIBUÍDOS (SBRC), 41. , 2023, Brasília/DF.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 211-224.
ISSN 2177-9384.
DOI: https://doi.org/10.5753/sbrc.2023.441.
