A Model for Traffic Forwarding through Service Function Chaining using Deep Reinforcement Learning Techniques
Resumo
The development of new communication networks to offer innovative services has increased the volume of data. With the introduction of Deep Reinforcement Learning and Service Function Chaining architecture, new research opportunities have emerged to propose solutions to the new challenges. This work proposes a model through computational simulations how these techniques can be applied. The model was evaluated using two variations of the Deep Q-Network algorithm over the CIC-Darknet dataset. Results showed that both variations are a promising mechanism to make the networks more autonomous and intelligent. to demonstrate
Referências
Akbari, I., Tahoun, E., Salahuddin, M.A., Limam, N., and Boutaba, R. (2020). ATMoS: Autonomous Threat Mitigation in SDN using Reinforcement Learning, in IEEE/IFIP Network Operations and Management Symposium, pages 1-9.
Arulkumaran, K., Deisenroth, M.P., Brundage, M., and Bharath, A.A. (2017). Deep Reinforcement Learning: A Brief Survey. IEEE Signal Processing Magazine, 34, pages 26-38.
ETSI ISG. (2014). ETSI GS NFV 002 V1.2.1. Network Functions Virtualisation (NFV); Architectural Framework. ETSI Group Specification.
Fu, X., Yu, F. R., Wang, J., Qi, Q. and Liao, J. (2019). Dynamic Service Function Chain Embedding for NFV-Enabled IoT: A Deep Reinforcement Learning Approach, in IEEE Transactions on Wireless Communications, vol. 19, no. 1, pages 507-519.
Haleplids, E., Pentikousis, K., Denazis, S., Salim, J. H., Meyer, D. and Koufopavlou, O. (2015). Software-Defined Networking (SDN): Layers and Architecture Terminology. RFC 7426, IETF.
Halpern, J. M. and Pignataro, C. (2015). Service Function Chaining (SFC) Architecture. RFC 7665, IETF.
He B., Wang, J., Qi, Q., Sun, H. and Liao, J. (2020). Towards Intelligent Provisioning of Virtualized Network Functions in Cloud of Things: A Deep Reinforcement Learning based Approach, in IEEE Transactions on Cloud Computing, vol., no. 01, pages 1-1.
ITU-T. (2018). Y.2242: Service Function Chain in Mobile Networks. ITU-T, Y series.
ITU-T. (2014). Y.3300: Framework of Software-Defined Networking. ITU-T, Y series.
Lashkari, A. H., Kaur, G., and Rahali A. (2020) “DIDarknet: A Contemporary Approach to Detect and Characterize the Darknet Traffic using Deep Image Learning”, 10th International Conference on Communication and Network Security, Tokyo, Japan, pages 1-13.
Li, G., Feng, B., Zhou, H., Zhang, Y., Sood, K., and Yu, S. (2020). Adaptive service in Computer in 5G using deep Q-learning, function chaining mappings Communications, vol. 152, pages 305-315.
Li, R., Zhao, Z., Sun, Q., Chih-lin, I., Yang, C., Chen, X., Zhao M. and Zhang, H. (2018). Deep Reinforcement Learning for Resource Management in Network Slicing, in IEEE Access, vol. 6, pages 74429-74441.
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y. and Kim, D. I. (2019). Applications of Deep Reinforcement Learning in Communications and Networking: A Survey, in IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pages 3133-3174.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M. A., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. (2015). Human-level control through deep reinforcement learning, in Nature, 518, pages 529-533.
Moustafa, N., and Slay, J. (2015). UNSW-NB15: a comprehensive data set for network in Military intrusion detection systems (UNSW-NB15 network data set), Communications and Information Systems Conference (MilCIS), pages 1-6.
Nguyen, T. T., and Reddi, V.J. (2019). Deep Reinforcement Learning for Cyber Security. ArXiv, abs/1906.05799.
Ning Z., Wang N. and R. Tafazolli. (2020). Deep Reinforcement Learning for NFV-based Service Function Chaining in Multi-Service Networks: Invited Paper, in IEEE 21st International Conference on High Performance Switching and Routing (HPSR), pages 1-6.
Pellegrini, J. and Wainer, J. (2007). Processos de Decisão de Markov: um tutorial. Revista de Informática Teórica e Aplicada; Vol. 14, No 2, páginas 133-179.
Peters, J. and Bagnell, J.A. (2010). Policy Gradient Methods. Encyclopedia of Machine Learning.
Quinn, P., Elzur, U. and Pignataro, C. (2018). Network Service Header (NSH). RFC 8300, IETF.
Restuccia, F., and Melodia, T. (2020). DeepWiERL: Bringing Deep Reinforcement Learning to the Internet of Self-Adaptive Things.
Riedmiller, M. (2005) Neural fitted Q iteration–first experiences with a data efficient neural reinforcement learning method, in European Conference on Machine Learning (ECML). Springer, pages 317–328.
Sethi, K., Kumar, R., Prajapati, N., and Bera, P. (2020). Deep Reinforcement Learning based Intrusion Detection System for Cloud Infrastructure, in International Conference on Communication Systems and Networks (COMSNETS), pages 1-6.
"Student" Gosset, W. S. (1908). "The probable error of a mean", in Biometrika. 6 (1) pages 1–25.
Taskin, Z.C. (2008). Tutorial Guide to Mixed-Integer Programming Models and Solution Techniques.
Xiao, Y., Zhang, Q., Liu, F., Wang, J., Zhao, M., Zhang, Z., and Zhang, J. (2019). NFVdeep: Adaptive Online Service Function Chain Deployment with Deep Reinforcement Learning. 2019 IEEE/ACM 27th International Symposium on Quality of Service (IWQoS), pages 1-10.
Zhang, J., Wang, Z., Ma, N., Huang, T., and Liu, Y. (2018). Enabling Efficient Service Function Chaining by Integrating NFV and SDN: Architecture, Challenges and Opportunities, in IEEE Network, vol. 32, no. 6 pages 152-159.
Arulkumaran, K., Deisenroth, M.P., Brundage, M., and Bharath, A.A. (2017). Deep Reinforcement Learning: A Brief Survey. IEEE Signal Processing Magazine, 34, pages 26-38.
ETSI ISG. (2014). ETSI GS NFV 002 V1.2.1. Network Functions Virtualisation (NFV); Architectural Framework. ETSI Group Specification.
Fu, X., Yu, F. R., Wang, J., Qi, Q. and Liao, J. (2019). Dynamic Service Function Chain Embedding for NFV-Enabled IoT: A Deep Reinforcement Learning Approach, in IEEE Transactions on Wireless Communications, vol. 19, no. 1, pages 507-519.
Haleplids, E., Pentikousis, K., Denazis, S., Salim, J. H., Meyer, D. and Koufopavlou, O. (2015). Software-Defined Networking (SDN): Layers and Architecture Terminology. RFC 7426, IETF.
Halpern, J. M. and Pignataro, C. (2015). Service Function Chaining (SFC) Architecture. RFC 7665, IETF.
He B., Wang, J., Qi, Q., Sun, H. and Liao, J. (2020). Towards Intelligent Provisioning of Virtualized Network Functions in Cloud of Things: A Deep Reinforcement Learning based Approach, in IEEE Transactions on Cloud Computing, vol., no. 01, pages 1-1.
ITU-T. (2018). Y.2242: Service Function Chain in Mobile Networks. ITU-T, Y series.
ITU-T. (2014). Y.3300: Framework of Software-Defined Networking. ITU-T, Y series.
Lashkari, A. H., Kaur, G., and Rahali A. (2020) “DIDarknet: A Contemporary Approach to Detect and Characterize the Darknet Traffic using Deep Image Learning”, 10th International Conference on Communication and Network Security, Tokyo, Japan, pages 1-13.
Li, G., Feng, B., Zhou, H., Zhang, Y., Sood, K., and Yu, S. (2020). Adaptive service in Computer in 5G using deep Q-learning, function chaining mappings Communications, vol. 152, pages 305-315.
Li, R., Zhao, Z., Sun, Q., Chih-lin, I., Yang, C., Chen, X., Zhao M. and Zhang, H. (2018). Deep Reinforcement Learning for Resource Management in Network Slicing, in IEEE Access, vol. 6, pages 74429-74441.
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y. and Kim, D. I. (2019). Applications of Deep Reinforcement Learning in Communications and Networking: A Survey, in IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pages 3133-3174.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M. A., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. (2015). Human-level control through deep reinforcement learning, in Nature, 518, pages 529-533.
Moustafa, N., and Slay, J. (2015). UNSW-NB15: a comprehensive data set for network in Military intrusion detection systems (UNSW-NB15 network data set), Communications and Information Systems Conference (MilCIS), pages 1-6.
Nguyen, T. T., and Reddi, V.J. (2019). Deep Reinforcement Learning for Cyber Security. ArXiv, abs/1906.05799.
Ning Z., Wang N. and R. Tafazolli. (2020). Deep Reinforcement Learning for NFV-based Service Function Chaining in Multi-Service Networks: Invited Paper, in IEEE 21st International Conference on High Performance Switching and Routing (HPSR), pages 1-6.
Pellegrini, J. and Wainer, J. (2007). Processos de Decisão de Markov: um tutorial. Revista de Informática Teórica e Aplicada; Vol. 14, No 2, páginas 133-179.
Peters, J. and Bagnell, J.A. (2010). Policy Gradient Methods. Encyclopedia of Machine Learning.
Quinn, P., Elzur, U. and Pignataro, C. (2018). Network Service Header (NSH). RFC 8300, IETF.
Restuccia, F., and Melodia, T. (2020). DeepWiERL: Bringing Deep Reinforcement Learning to the Internet of Self-Adaptive Things.
Riedmiller, M. (2005) Neural fitted Q iteration–first experiences with a data efficient neural reinforcement learning method, in European Conference on Machine Learning (ECML). Springer, pages 317–328.
Sethi, K., Kumar, R., Prajapati, N., and Bera, P. (2020). Deep Reinforcement Learning based Intrusion Detection System for Cloud Infrastructure, in International Conference on Communication Systems and Networks (COMSNETS), pages 1-6.
"Student" Gosset, W. S. (1908). "The probable error of a mean", in Biometrika. 6 (1) pages 1–25.
Taskin, Z.C. (2008). Tutorial Guide to Mixed-Integer Programming Models and Solution Techniques.
Xiao, Y., Zhang, Q., Liu, F., Wang, J., Zhao, M., Zhang, Z., and Zhang, J. (2019). NFVdeep: Adaptive Online Service Function Chain Deployment with Deep Reinforcement Learning. 2019 IEEE/ACM 27th International Symposium on Quality of Service (IWQoS), pages 1-10.
Zhang, J., Wang, Z., Ma, N., Huang, T., and Liu, Y. (2018). Enabling Efficient Service Function Chaining by Integrating NFV and SDN: Architecture, Challenges and Opportunities, in IEEE Network, vol. 32, no. 6 pages 152-159.
Publicado
29/11/2021
Como Citar
ARAÚJO JÚNIOR, Silvio Romero de; BIANCHI, Reinaldo A. C..
A Model for Traffic Forwarding through Service Function Chaining using Deep Reinforcement Learning Techniques. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 18. , 2021, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 619-630.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2021.18289.
