On Enhancing Network Throughput using Reinforcement Learning in Sliced Testbeds
Resumo
Novel applications demand high throughput, low latency, and high reliability connectivity and still pose significant challenges to slicing orchestration architectures. The literature explores network slicing techniques that employ canonical methods, artificial intelligence, and combinatorial optimization to address errors and ensure throughput for network slice data plane. This paper introduces the Enhanced Mobile Broadband (eMBB)-Agent as a new approach that uses Reinforcement Learning (RL) in a vertical application to enhance network slicing throughput to fit Service-Level Agreements (SLAs). The eMBB-Agent analyzes application transmission variables and proposes actions within a discrete space to adjust the reception window using a Deep Q-Network (DQN). This paper also presents experimental results that examine the impact of factors such as the channel error rate, DQN model layers, and learning rate on model convergence and achieved throughput, providing insights on embedding intelligence in network slicing.Referências
Aripin, N. M., Zulkifli, T., and Radzi, N. A. M. (2023). Performance Analysis of 5G Network Slicing for Hospital of the Future. In 2023 IEEE 13th Symposium on Computer Applications & Industrial Electronics (ISCAIE), pages 18–21.
Beig, E. F. G. M., Daneshjoo, P., Rezaei, S., Movassagh, A. A., Karimi, R., and Qin, Y. (2018). Mptcp throughput enhancement by q-learning for mobile devices. In 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), pages 1171–1176, Conference. IEEE.
Brilhante, D. d. S., Manjarres, J. C., Moreira, R., de Oliveira Veiga, L., de Rezende, J. F., Müller, F., Klautau, A., Leonel Mendes, L., and P. de Figueiredo, F. A. (2023). A Literature Survey on AI-Aided Beamforming and Beam Management for 5G and 6G Systems. Sensors, 23(9).
Gawlowicz, P. and Zubow, A. (2018). ns3-gym: Extending openai gym for networking research. CoRR, abs/1810.03943.
Khan, B. S., Jangsher, S., Ahmed, A., and Al-Dweik, A. (2022). URLLC and eMBB in 5G Industrial IoT: A Survey. IEEE Open Journal of the Communications Society, 3:1134–1163.
Li, W., Zhang, H., Gao, S., Xue, C., Wang, X., and Lu, S. (2019a). SmartCC: A Reinforcement Learning Approach for Multipath TCP Congestion Control in Heterogeneous Networks. IEEE Journal on Selected Areas in Communications, 37(11):2621–2633.
Li, W., Zhou, F., Chowdhury, K. R., and Meleis, W. (2019b). QTCP: Adaptive Congestion Control with Reinforcement Learning. IEEE Transactions on Network Science and Engineering, 6(3):445–458.
Liu, Y., Clerckx, B., and Popovski, P. (2023). Network Slicing for eMBB, URLLC, and mMTC: An Uplink Rate-Splitting Multiple Access Approach. IEEE Transactions on Wireless Communications, pages 1–1.
Moreira, R., Rodrigues Moreira, L. F., and de Oliveira Silva, F. (2023). An intelligent network monitoring approach for online classification of Darknet traffic. Computers and Electrical Engineering, 110:108852.
Moreira, R., Rosa, P. F., Aguiar, R. L. A., and de Oliveira Silva, F. (2021). Deploying Scalable and Stable XDP-Based Network Slices Through NASOR Framework for Low-Latency Applications. In Barolli, L., Woungang, I., and Enokido, T., editors, Advanced Information Networking and Applications, pages 715–726, Cham. Springer International Publishing.
Ojijo, M. O. and Falowo, O. E. (2020). A Survey on Slice Admission Control Strategies and Optimization Schemes in 5G Network. IEEE Access, 8:14977–14990.
Siddiqi, S. J., Naeem, F., Khan, S., Khan, K. S., and Tariq, M. (2022). Towards AI-enabled traffic management in multipath TCP: A survey. Computer Communications, 181:412–427.
Tang, F., Fadlullah, Z. M., Mao, B., and Kato, N. (2018). An Intelligent Traffic Load Prediction-Based Adaptive Channel Assignment Algorithm in SDN-IoT: A Deep Learning Approach. IEEE Internet of Things Journal, 5(6):5141–5154.
Vieira, F. H. T. and Garcez, S. G. (2011). Estimação de probabilidade de perda de dados em redes através de modelagem multifractal de tráfego e teoria de muitas fontes. Revista de Informática Teórica e Aplicada, 18(1):13–30.
Zhang, H., Li, W., Gao, S., Wang, X., and Ye, B. (2019). ReLeS: A Neural Adaptive Multipath Scheduler based on Deep Reinforcement Learning. In IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pages 1648–1656, INFOCOM. IEEE.
Beig, E. F. G. M., Daneshjoo, P., Rezaei, S., Movassagh, A. A., Karimi, R., and Qin, Y. (2018). Mptcp throughput enhancement by q-learning for mobile devices. In 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), pages 1171–1176, Conference. IEEE.
Brilhante, D. d. S., Manjarres, J. C., Moreira, R., de Oliveira Veiga, L., de Rezende, J. F., Müller, F., Klautau, A., Leonel Mendes, L., and P. de Figueiredo, F. A. (2023). A Literature Survey on AI-Aided Beamforming and Beam Management for 5G and 6G Systems. Sensors, 23(9).
Gawlowicz, P. and Zubow, A. (2018). ns3-gym: Extending openai gym for networking research. CoRR, abs/1810.03943.
Khan, B. S., Jangsher, S., Ahmed, A., and Al-Dweik, A. (2022). URLLC and eMBB in 5G Industrial IoT: A Survey. IEEE Open Journal of the Communications Society, 3:1134–1163.
Li, W., Zhang, H., Gao, S., Xue, C., Wang, X., and Lu, S. (2019a). SmartCC: A Reinforcement Learning Approach for Multipath TCP Congestion Control in Heterogeneous Networks. IEEE Journal on Selected Areas in Communications, 37(11):2621–2633.
Li, W., Zhou, F., Chowdhury, K. R., and Meleis, W. (2019b). QTCP: Adaptive Congestion Control with Reinforcement Learning. IEEE Transactions on Network Science and Engineering, 6(3):445–458.
Liu, Y., Clerckx, B., and Popovski, P. (2023). Network Slicing for eMBB, URLLC, and mMTC: An Uplink Rate-Splitting Multiple Access Approach. IEEE Transactions on Wireless Communications, pages 1–1.
Moreira, R., Rodrigues Moreira, L. F., and de Oliveira Silva, F. (2023). An intelligent network monitoring approach for online classification of Darknet traffic. Computers and Electrical Engineering, 110:108852.
Moreira, R., Rosa, P. F., Aguiar, R. L. A., and de Oliveira Silva, F. (2021). Deploying Scalable and Stable XDP-Based Network Slices Through NASOR Framework for Low-Latency Applications. In Barolli, L., Woungang, I., and Enokido, T., editors, Advanced Information Networking and Applications, pages 715–726, Cham. Springer International Publishing.
Ojijo, M. O. and Falowo, O. E. (2020). A Survey on Slice Admission Control Strategies and Optimization Schemes in 5G Network. IEEE Access, 8:14977–14990.
Siddiqi, S. J., Naeem, F., Khan, S., Khan, K. S., and Tariq, M. (2022). Towards AI-enabled traffic management in multipath TCP: A survey. Computer Communications, 181:412–427.
Tang, F., Fadlullah, Z. M., Mao, B., and Kato, N. (2018). An Intelligent Traffic Load Prediction-Based Adaptive Channel Assignment Algorithm in SDN-IoT: A Deep Learning Approach. IEEE Internet of Things Journal, 5(6):5141–5154.
Vieira, F. H. T. and Garcez, S. G. (2011). Estimação de probabilidade de perda de dados em redes através de modelagem multifractal de tráfego e teoria de muitas fontes. Revista de Informática Teórica e Aplicada, 18(1):13–30.
Zhang, H., Li, W., Gao, S., Wang, X., and Ye, B. (2019). ReLeS: A Neural Adaptive Multipath Scheduler based on Deep Reinforcement Learning. In IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pages 1648–1656, INFOCOM. IEEE.
Publicado
24/05/2024
Como Citar
MONTEIRO, Daniel Pereira; SAAR, Lucas Nardelli de Freitas Botelho; MOREIRA, Larissa Ferreira Rodrigues; MOREIRA, Rodrigo.
On Enhancing Network Throughput using Reinforcement Learning in Sliced Testbeds. In: WORKSHOP DE PESQUISA EXPERIMENTAL DA INTERNET DO FUTURO (WPEIF), 15. , 2024, Niterói/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 1-7.
ISSN 2595-2692.
DOI: https://doi.org/10.5753/wpeif.2024.2094.
