Analysis of Network Performance over Deep Reinforcement Learning Control Loops for Industry 4.0
Resumo
Dentre os três principais casos de uso das redes 5G está a comunicação ultra-confiável de baixa latência, que inclui a operação remota para a Indústria 4.0. Nesse tipo de aplicação, robôs autônomos são capazes de operar sem intervenção humana, guiados por sistemas de inteligência artificial executados em nuvem. Este artigo investiga dois diferentes aspectos: (i) a performance de um modelo pré-treinado em condições de rede ideais; e (ii) como variações de desempenho da rede influenciam o desempenho dos modelos no ambiente. Os resultados mostram a influência de diferentes condições de qualidade da rede no desempenho do agente, assim como o benefício de utilizar agentes pré-treinados em condições ideais.
Referências
Akpakwu, G. A., Silva, B. J., Hancke, G. P., and Abu-Mahfouz, A. M. (2017). A survey on 5g networks for the internet of things: Communication technologies and challenges. IEEE access, 6:3619–3647.
Atzori, L., Iera, A., and Morabito, G. (2010). The internet of things: A survey. Computer Networks, 54(15):2787–2805.
Beltrão, S. R. B. B. and de Oliveira Pires, A. A. (2019). Generation of corporate intelligence in industry 4.0, through the combination of professional drones and artificial intelligence. case study applied to a coal recovery boiler. Brazilian Journal of Technology, 2(4):946–966.
Buchner, R., Wurhofer, D., Weiss, A., and Tscheligi, M. (2012). User experience of industrial robots over time. In 2012 7th ACM/IEEE International Conference on HumanRobot Interaction (HRI), pages 115–116.
Chen, Y., Farley, T., and Ye, N. (2004). Qos requirements of network applications on the internet. Information Knowledge Systems Management, 4(1):55–76.
Cheng, J., Chen, W., Tao, F., and Lin, C.-L. (2018). Industrial iot in 5g environment towards smart manufacturing. Journal of Industrial Information Integration, 10:10– 19.
Civerchia, F., Giannone, F., Kondepu, K., Castoldi, P., Valcarenghi, L., Bragagnini, A., Gatti, F., Napolitano, A., and Borromeo, J. C. (2020). Remote control of a robot rover combining 5g, ai, and gpu image processing at the edge. In Optical Fiber Communication Conference, pages M3Z–10. Optical Society of America.
ETSI (2020). Service requirements for cyber-physical control applications in vertical domains. Acessed in 01.10.2022.
Faheem, M., Shah, S., Butt, R., Raza, B., Anwar, M., Ashraf, M., Ngadi, M., and Gungor, V. (2018). Smart grid communication and information technologies in the perspective of industry 4.0: Opportunities and challenges. Computer Science Review, 30:1–30.
Harris, C. J. (1994). Advances in intelligent control. CRC Press.
Huang, G., Rao, P. S., Wu, M.-H., Qian, X., Nof, S. Y., Ramani, K., and Quinn, A. J. (2020). Vipo: Spatial-visual programming with functions for robot-iot workflows. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–13.
Jiang, Z., Fu, S., Zhou, S., Niu, Z., Zhang, S., and Xu, S. (2020). Ai-assisted low information latency wireless networking. IEEE Wireless Communications, 27(1):108–115.
Kunst, R., Avila, L., Binotto, A., Pignaton, E., Bampi, S., and Rochol, J. (2019). Improving devices communication in industry 4.0 wireless networks. Engineering Applications of Artificial Intelligence, 83:1–12.
Liberato, A., Martinello, M., Gomes, R. L., Beldachi, A. F., Salas, E., Villaca, R., Ribeiro, M. R., Kondepu, K., Kanellos, G., Nejabati, R., et al. (2018). Rdna: Residue-defined networking architecture enabling ultra-reliable low-latency datacenters. IEEE Transactions on Network and Service Management, 15(4):1473–1487.
Liu, Q., Zoppi, S., Tan, G., Kellerer, W., and Steinbach, E. (2017). Quality-of-controldriven uplink scheduling for networked control systems running over 5g communication networks. In 2017 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE), pages 1–6.
Lobbrecht, A. H. and Solomatine, D. P. (2002). Machine learning in real-time control of water systems. Urban Water, 4(3):283–289.
Luo, G., Yuan, Q., Li, J., Wang, S., and Yang, F. (2021). Artificial intelligence powered mobile networks: From cognition to decision. arXiv preprint arXiv:2112.04263.
Marques, P., do Carmo, A. P., Frascolla, V., Silva, C., Sena, E. D., Braga, R., Pinheiro, J., Astudillo, C. A., de Andrade, T. P., Gama, E. S., et al. (2018). Optical and wireless network convergence in 5g systems–an experimental approach. In 2018 IEEE 23rd International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), pages 1–5. IEEE.
Nakimuli, W., Garcia-Reinoso, J., Sierra-Garcia, J. E., Serrano, P., and Fernández, I. Q. (2021). Deployment and evaluation of an industry 4.0 use case over 5g. IEEE Communications Magazine, 59(7):14–20.
Nasrallah, A., Thyagaturu, A. S., Alharbi, Z., Wang, C., Shao, X., Reisslein, M., and ElBakoury, H. (2018). Ultra-low latency (ull) networks: The ieee tsn and ietf detnet standards and related 5g ull research. IEEE Communications Surveys & Tutorials, 21(1):88–145.
Papcun, P., Kajáti, E., and Koziorek, J. (2018). Human machine interface in concept of industry 4.0. In 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA), pages 289–296.
Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., and Dormann, N. (2021).
Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22(268):1–8.
Rodriguez, I., Mogensen, R. S., Fink, A., Raunholt, T., Markussen, S., Christensen, P. H., Berardinelli, G., Mogensen, P., Schou, C., and Madsen, O. (2021). An experimental framework for 5g wireless system integration into industry 4.0 applications. Energies, 14(15):4444.
Simonini, T. (2018). An intro to advantage actor critic methods: let’s play sonic the hedgehog! [link]. Acessed in 06.02.2022.
Yang, S.-Y., Jin, S.-M., and Kwon, S.-K. (2008). Remote control system of industrial field robot. In 2008 6th IEEE International Conference on Industrial Informatics, pages 442–447. IEEE.
Atzori, L., Iera, A., and Morabito, G. (2010). The internet of things: A survey. Computer Networks, 54(15):2787–2805.
Beltrão, S. R. B. B. and de Oliveira Pires, A. A. (2019). Generation of corporate intelligence in industry 4.0, through the combination of professional drones and artificial intelligence. case study applied to a coal recovery boiler. Brazilian Journal of Technology, 2(4):946–966.
Buchner, R., Wurhofer, D., Weiss, A., and Tscheligi, M. (2012). User experience of industrial robots over time. In 2012 7th ACM/IEEE International Conference on HumanRobot Interaction (HRI), pages 115–116.
Chen, Y., Farley, T., and Ye, N. (2004). Qos requirements of network applications on the internet. Information Knowledge Systems Management, 4(1):55–76.
Cheng, J., Chen, W., Tao, F., and Lin, C.-L. (2018). Industrial iot in 5g environment towards smart manufacturing. Journal of Industrial Information Integration, 10:10– 19.
Civerchia, F., Giannone, F., Kondepu, K., Castoldi, P., Valcarenghi, L., Bragagnini, A., Gatti, F., Napolitano, A., and Borromeo, J. C. (2020). Remote control of a robot rover combining 5g, ai, and gpu image processing at the edge. In Optical Fiber Communication Conference, pages M3Z–10. Optical Society of America.
ETSI (2020). Service requirements for cyber-physical control applications in vertical domains. Acessed in 01.10.2022.
Faheem, M., Shah, S., Butt, R., Raza, B., Anwar, M., Ashraf, M., Ngadi, M., and Gungor, V. (2018). Smart grid communication and information technologies in the perspective of industry 4.0: Opportunities and challenges. Computer Science Review, 30:1–30.
Harris, C. J. (1994). Advances in intelligent control. CRC Press.
Huang, G., Rao, P. S., Wu, M.-H., Qian, X., Nof, S. Y., Ramani, K., and Quinn, A. J. (2020). Vipo: Spatial-visual programming with functions for robot-iot workflows. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–13.
Jiang, Z., Fu, S., Zhou, S., Niu, Z., Zhang, S., and Xu, S. (2020). Ai-assisted low information latency wireless networking. IEEE Wireless Communications, 27(1):108–115.
Kunst, R., Avila, L., Binotto, A., Pignaton, E., Bampi, S., and Rochol, J. (2019). Improving devices communication in industry 4.0 wireless networks. Engineering Applications of Artificial Intelligence, 83:1–12.
Liberato, A., Martinello, M., Gomes, R. L., Beldachi, A. F., Salas, E., Villaca, R., Ribeiro, M. R., Kondepu, K., Kanellos, G., Nejabati, R., et al. (2018). Rdna: Residue-defined networking architecture enabling ultra-reliable low-latency datacenters. IEEE Transactions on Network and Service Management, 15(4):1473–1487.
Liu, Q., Zoppi, S., Tan, G., Kellerer, W., and Steinbach, E. (2017). Quality-of-controldriven uplink scheduling for networked control systems running over 5g communication networks. In 2017 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE), pages 1–6.
Lobbrecht, A. H. and Solomatine, D. P. (2002). Machine learning in real-time control of water systems. Urban Water, 4(3):283–289.
Luo, G., Yuan, Q., Li, J., Wang, S., and Yang, F. (2021). Artificial intelligence powered mobile networks: From cognition to decision. arXiv preprint arXiv:2112.04263.
Marques, P., do Carmo, A. P., Frascolla, V., Silva, C., Sena, E. D., Braga, R., Pinheiro, J., Astudillo, C. A., de Andrade, T. P., Gama, E. S., et al. (2018). Optical and wireless network convergence in 5g systems–an experimental approach. In 2018 IEEE 23rd International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), pages 1–5. IEEE.
Nakimuli, W., Garcia-Reinoso, J., Sierra-Garcia, J. E., Serrano, P., and Fernández, I. Q. (2021). Deployment and evaluation of an industry 4.0 use case over 5g. IEEE Communications Magazine, 59(7):14–20.
Nasrallah, A., Thyagaturu, A. S., Alharbi, Z., Wang, C., Shao, X., Reisslein, M., and ElBakoury, H. (2018). Ultra-low latency (ull) networks: The ieee tsn and ietf detnet standards and related 5g ull research. IEEE Communications Surveys & Tutorials, 21(1):88–145.
Papcun, P., Kajáti, E., and Koziorek, J. (2018). Human machine interface in concept of industry 4.0. In 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA), pages 289–296.
Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., and Dormann, N. (2021).
Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22(268):1–8.
Rodriguez, I., Mogensen, R. S., Fink, A., Raunholt, T., Markussen, S., Christensen, P. H., Berardinelli, G., Mogensen, P., Schou, C., and Madsen, O. (2021). An experimental framework for 5g wireless system integration into industry 4.0 applications. Energies, 14(15):4444.
Simonini, T. (2018). An intro to advantage actor critic methods: let’s play sonic the hedgehog! [link]. Acessed in 06.02.2022.
Yang, S.-Y., Jin, S.-M., and Kwon, S.-K. (2008). Remote control system of industrial field robot. In 2008 6th IEEE International Conference on Industrial Informatics, pages 442–447. IEEE.
Publicado
23/05/2022
Como Citar
BERNARDO, Guilherme T. T.; MIRANDA JR., Gilson; MACEDO, Daniel F..
Analysis of Network Performance over Deep Reinforcement Learning Control Loops for Industry 4.0. In: SIMPÓSIO BRASILEIRO DE REDES DE COMPUTADORES E SISTEMAS DISTRIBUÍDOS (SBRC), 40. , 2022, Fortaleza.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 1-14.
ISSN 2177-9384.
DOI: https://doi.org/10.5753/sbrc.2022.221898.
