A Path-Aware Routing for Data Intensive Science: Proposal, Deployment and Evaluation in High-Performance Testbed

• Daniel Ventorim UFES / IFES http://orcid.org/0009-0000-1159-085X
• Everson S. Borges UFES / IFES https://orcid.org/0000-0002-8521-829X
• Rafael S. Guimarães IFES https://orcid.org/0000-0001-6856-9576
• Magnos Martinello UFES https://orcid.org/0000-0002-8111-1719
• Moises R. N. Ribeiro UFES http://orcid.org/0000-0003-1434-6566
• Cristina K. Dominicini IFES https://orcid.org/0000-0002-7803-1830
• Marcos Schwarz RNP https://orcid.org/0000-0002-3461-3548
• Bruno Xavier IFES
• Jeronimo Bezerra FIU https://orcid.org/0000-0002-8861-0035
• Mariam Kiran ORNL http://orcid.org/0000-0001-8455-9546
• Harvey Newman Caltech https://orcid.org/0000-0003-0964-1480

DOI: https://doi.org/10.5753/wgrs.2025.8769

Abstract

This paper introduces a path-aware architecture designed to enhance traffic engineering through path controllability and visibility for Data-Intensive Science (DIS). The approach relies on a stateless core network architecture that leverages line-rate packet forwarding reusing the cyclic redundancy check (CRC) implemented with P4 language. A path-aware approach was developed to enable path selection and hardware network metric collection. As a proof of concept, our approach was deployed in a 100+ Gbps multipath intradatacenter topology. The evaluation includes experiments on i) a 100+ Gbps multipath intra-datacenter topology and ii) inter-datacenter networking across a Pan-American hybrid infrastructure connecting Vitória, ES, Brazil, and Atlanta, GA, USA. The traffic steering is demonstrated for intra-datacenter at the Caltech testbed, where concurrent flows achieve approximately 40 Gbps each at line rate. Inter-datacenter communication is evaluated over a 10,000 km link as part of a record-breaking 10 Tbps traffic stress at Supercomputing 2024.

Keywords: programable networking, DIS, path-aware, routing, PolKA, source routing, RNS

References

Al-Najjar, A. et al. (2024). Framework for integrating machine learning methods for path-aware source routing. In SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 829–838. IEEE.

Anderson, T. et al. (2014). A brief overview of the nebula future internet architecture. SIGCOMM Comput. Commun. Rev., 44(3):81–86.

Barisits, M., Beermann, T., Berghaus, F., Bockelman, B., Bogado, J., Cameron, D., Christidis, D., Ciangottini, D., Dimitrov, G., Elsing, M., et al. (2019). Rucio: Scientific data management. Computing and Software for Big Science, 3:1–19.

Barrera, D., Chuat, L., Perrig, A., Reischuk, R. M., and Szalachowski, P. (2017). The scion internet architecture. Commun. ACM, 60(6):56–65.

Borges, E., Pontes, E., Mate, C., Loui, F., Martinello, M., and Ribeiro, M. (2022). Freerouter in a nutshell: A protocol and routing platform for open and portable carrier-class testbeds. In Anais do I Workshop de Testbeds, pages 36–46, Porto Alegre, RS, Brasil. SBC.

Dominicini, C. et al. (2020a). Polka: Polynomial key-based architecture for source routing in network fabrics. In 2020 6th IEEE Conference on Network Softwarization (NetSoft), pages 326–334. IEEE.

Dominicini, C., Guimarães, R., Mafioletti, D., Martinello, M., Ribeiro, M. R., Villaça, R., Loui, F., Ortiz, J., Slyne, F., Ruffini, M., et al. (2021). Deploying polka source routing in P4 switches. In 2021 International Conference on Optical Network Design and Modeling (ONDM), pages 1–3. IEEE.

Dominicini, C., Mafioletti, D., Locateli, A. C., Villaca, R., Martinello, M., Ribeiro, M., and Gorodnik, A. (2020b). Polka: Polynomial key-based architecture for source routing in network fabrics. In 2020 6th IEEE Conference on Network Softwarization (NetSoft), pages 326–334. IEEE.

Dominicini, C. K. et al. (2020c). KeySFC: Traffic steering using strict source routing for dynamic and efficient network orchestration. Computer Networks, 167:106975.

Dunefsky, J. et al. (2022). Transport control networking: Optimizing efficiency and control of data transport for data-intensive networks. In Proceedings of the ACM SIGCOMM Workshop on Network-Application Integration, NAI ’22, pages 60–66, New York, NY, USA. Association for Computing Machinery.

Gomes, R. R. et al. (2016). KAR: Key-for-any-route, a resilient routing system. In 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshop, pages 120–127.

Guimarães, R. S. et al. (2022). M-polka: Multipath polynomial key-based source routing for reliable communications. IEEE Transactions on Network and Service Management, pages 1–1.

GÉANT GP4L (2024). Gp4l documentation. Accessed: Jan 2025.

Harutyunyan, H., Swenson, C., and Newman, H. B. (2009). FDT: A high-speed data transfer tool for scientific applications. International Journal of High Performance Computing Applications, 23(3):254–267.

InMon Corporation (2023). InMon Traffic Monitor Tool. Accessed: Jan 2025.

Jin, X., Farrington, N., and Rexford, J. (2016). Your data center switch is trying too hard. In Proceedings of the Symposium on SDN Research, pages 1–6.

Johnston, W., Dart, E., and Tierney, B. Addressing the problem of data mobility for data-intensive science. In Proceedings of the Eighth International Conference on Engineering Computational Technology.

Jyothi, S. A., Dong, M., and Godfrey, P. B. (2015). Towards a flexible data center fabric with source routing. In Proceedings of the 1st ACM SIGCOMM Symposium on Software Defined Networking Research, pages 1–8.

Liberato, A. et al. (2018). RDNA: Residue-Defined Networking Architecture Enabling Ultra-Reliable Low-Latency Datacenters. IEEE TNSM.

Martinello, M., Ribeiro, M. R., de Oliveira, R. E. Z., and de Angelis Vitoi, R. (2014). Keyflow: A prototype for evolving SDN toward core network fabrics. IEEE Network, 28(2):12–19.

Monga, I., Guok, C., MacAuley, J., Sim, A., Newman, H., Balcas, J., DeMar, P., Winkler, L., Lehman, T., and Yang, X. (2020). Software-defined network for end-to-end networked science at the exascale. Future Generation Computer Systems, 110:181–201.

Newman, H., Mughal, A., Kcira, D., Legrand, I., Voicu, R., and Bunn, J. (2015). High speed scientific data transfers using software defined networking. In Proceedings of the Second Workshop on Innovating the Network for Data-Intensive Science, pages 1–9.

Newman, H. B., Ellisman, M. H., and Orcutt, J. A. (2003). Data-intensive e-science frontier research. Commun. ACM, 46(11):68–77.

Ren, Y. et al. (2017). Flowtable-free routing for data center networks: A software-defined approach. In GLOBECOM 2017 - 2017 IEEE Global Communications Conference, pages 1–6.

Scherrer, S., Legner, M., Perrig, A., and Schmid, S. (2021). Enabling novel interconnection agreements with path-aware networking architectures. In Proceedings of the 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pages 1–12. IEEE.

Sunshine, C. A. (1977). Source routing in computer networks. ACM SIGCOMM Computer Communication Review, 7(1):29–33.

Trammell, B. (2022). Current open questions in path-aware networking. RFC 9217, IRTF.

Ventre, P. L. et al. (2020). Segment routing: A comprehensive survey of research activities, standardization efforts, and implementation results. IEEE Communications Surveys & Tutorials, 23(1):182–221.

Viswanathan, A., Rosen, E. C., and Callon, R. (2001). Multiprotocol Label Switching Architecture. RFC 3031.

Wessing, H. et al. (2002). Novel scheme for packet forwarding without header modifications in optical networks. Journal of Lightwave Technology, 20(8):1277–1283.

Zurawski, J., Brown, B., Carder, D., Colby, E., Dart, E., Miller, K., Patwa, A., Robinson, K., Rotman, L., and Wiedlea, A. (2021). 2020 high energy physics network requirements review final report.