Uplink Resource Allocation Based on Reinforcement Learning Considering Power and Delay Optimization for 5G Networks with D2D Communications

• Marcus V. G. Ferreira UFG
• Leonardo A. Melo UFG
• Flávio H. T. Vieira UFG

DOI: https://doi.org/10.5753/semish.2025.9359

Abstract

In this paper, the problem of resource allocation for devices is investigated in a multi-sharing uplink scenario within cyclic prefix orthogonal frequency-division multiplexing (CP-OFDM) and millimeter-waves (mmWaves)-based wireless networks, incorporating device-to-device (D2D) communications. Specifically, a power and delay optimization uplink resource allocation algorithm with reinforcement learning (PDO-URA-RL) is proposed, divided into two stages. First, network resources are allocated to cellular user equipments (CUEs) in terms of power and transmission rate through an approach aimed at maximizing throughput. Subsequently, idle resources are allocated with a focus on minimizing delay. Computational simulations are conducted in a 5G-oriented communication scenario, leveraging mmWaves frequencies above 6 GHz, while comparing the performance with other literature algorithms in terms of quality of service (QoS) parameters, such as throughput and delay.

References

mmwave: o que são as ondas milimétricas que farão o 5g funcionar em altíssimas frequências. [link]. Jun 30, 2021.

University of waikato - wand network research group: Series of real tcp/ip traffic. [link]. Jun 24, 2021.

(2017). 3gpp tr 38.901 version 14.0.0 release 14. 5g; study on channel model for frequencies from 0.5 to 100 ghz.

(2018a). 3gpp ts 38.104 version 15.2.0 release 15. 5g; nr; base station (bs) radio transmission and reception.

(2018b). 3gpp ts 38.211 version 15.2.0 release 15. 5g; nr; physical channels and modulation.

Belgiovine, M., Sankhe, K., Bocanegra, C., Roy, D., and Chowdhury, K. R. (2021). Deep learning at the edge for channel estimation in beyond-5g massive mimo. IEEE Wireless Communications, 28(2):19–25.

Ciou, S., Kao, J., Lee, C. Y., and Chen, K. (2015). Multi-sharing resource allocation for device-to-device communication underlaying 5g mobile networks. In 2015 IEEE 26th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), pages 1509–1514.

Ferreira, M. V., Vieira, F. H., and Álisson A. Cardoso (2022). Power and delay optimization based uplink resource allocation for wireless networks with device-to-device communications. Computer Communications, 194:226–239.

Ferreira, M. V. G. and Vieira, F. H. T. (2021). Delay minimization based uplink resource allocation for device-to-device communications considering mmwave propagation. PeerJ Computer Science, 7:e462.

Karipidis, K., Mate, R., Urban, D., Tinker, R., and Wood, A. (2021). 5g mobile networks and health - a state-of-the-science review of the research into low-level rf fields above 6 ghz. Journal of Exposure Science and Environmental Epidemiology, 31:585–605.

Sreedevi, A. and Rama Rao, T. (2019). Reinforcement learning algorithm for 5g indoor device-to-device communications. Transactions on Emerging Telecommunications Technologies, 30(9):e3670.

Sun, H., Sheng, M., Wang, X., Zhang, Y., Liu, J., and Wang, K. (2013). Resource allocation for maximizing the device-to-device communications underlaying lte-advanced networks. In 2013 IEEE/CIC International Conference on Communications in China - Workshops (CIC/ICCC), pages 60–64.