Distributed Reinforcement Learning for Throughput Optimization in TSCH Networks

• Vanessa N. Silva UFMG
• Guilherme Milanez UFMG
• Marcos A. M. Vieira UFMG
• Luiz Filipe M. Vieira UFMG
• José Augusto M. Nacif UFV

DOI: https://doi.org/10.5753/sbrc.2026.19256

Abstract

The Time Slotted Channel Hopping (TSCH) protocol has been widely adopted in industrial and Internet of Things (IoT) scenarios due to its reliability and energy efficiency. However, despite the existence of several scheduling mechanisms focused on robustness, there remains a gap regarding throughput optimization, one of the most relevant metrics in network performance evaluation. In this context, this work proposes a distributed reinforcement learning approach based on Q-learning for throughput optimization in TSCH networks, in which each sensor node acts as an autonomous agent that locally learns the best transmission slot allocation over time. The solution was implemented in Contiki-NG and employs a multi-criteria reward function capable of capturing different aspects of network performance, without requiring centralized control or excessive information exchange among nodes, thereby reducing communication overhead. The learning process is continuous, enabling dynamic adaptation to network conditions, and the obtained results indicate the potential of reinforcement learning as an effective mechanism for adaptive scheduling in TSCH networks.

References

Ara, T. and Liscano, R. (2024). Optimal slot utilization in ieee 802.15.4e tsch networks using reinforcement learning. In 2024 IEEE 35th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pages 1–7.

Ben Yaala, S., Ben Yaala, S., and Bouallegue, R. (2025). Optimizing tsch scheduling for iiot networks using reinforcement learning. Technologies, 13(9).

Cardel, V. S. (2025). A systematic review and q-learning-based design of scheduling functions for 6tisch networks. Dissertação de mestrado, Instituto de Computação, Universidade Federal da Bahia, Salvador, Bahia, Brasil. Mestrado em Ciência da Computação.

Chang, T., Vučinić, M., Vilajosana, X., Duquennoy, S., and Dujovne, D. R. (2021). 6TiSCH Minimal Scheduling Function (MSF). RFC 9033.

dos Santos Ribeiro, N., Vieira, M. A., Vieira, L. F., and Gnawali, O. (2022). Splitpath: High throughput using multipath routing in dual-radio wireless sensor networks. Computer Networks, 207:108832.

Duquennoy, S., Elsts, A., Al Nahas, B., and Oikonomou, G. (2017). Tsch and 6tisch for contiki: Challenges, design and evaluation.

Milanez, G. A., Vieira, M. A., Vieira, L. F., and Nacif, J. A. M. (2023). Variban: A variable bandwidth channel allocation algorithm for ieee 802.15. 4e-based networks. Computer Networks, 231:109774.

Mora, A., Bujari, A., and Bellavista, P. (2024). Enhancing generalization in federated learning with heterogeneous data: A comparative literature review. Future Generation Computer Systems, 157:1–15.

Naparstek, O. and Cohen, K. (2019). Deep multi-user reinforcement learning for distributed dynamic spectrum access. IEEE Transactions on Wireless Communications, 18(1):310–323.

Palattella, M. R., Accettura, N., Dohler, M., Grieco, L. A., and Boggia, G. (2012). Traffic aware scheduling algorithm for reliable low-power multi-hop ieee 802.15. 4e networks. In 2012 IEEE 23rd International Symposium on Personal, Indoor and Mobile Radio Communications-(PIMRC), pages 327–332. IEEE.

Pratama, Y. H., Chung, S.-H., and Fawwaz, D. Z. (2024). Low-latency and q-learning-based distributed scheduling function for dynamic 6tisch networks. IEEE Access, 12:49694–49707.

Salih, K. O. M., Rashid, T. A., Radovanovic, D., and Bacanin, N. (2022). A comprehensive survey on the internet of things with the industrial marketplace. Sensors, 22(3).

Savaglio, C., Pace, P., Aloi, G., Liotta, A., and Fortino, G. (2019). Lightweight reinforcement learning for energy efficient communications in wireless sensor networks. IEEE Access, 7:29355–29364.

Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press, 2nd edition.

Vilajosana, X., Pister, K., and Watteyne, T. (2017). Minimal IPv6 over the TSCH Mode of IEEE 802.15.4e (6TiSCH) Configuration. RFC 8180.