AFP: A Feedback-Guided Microservice Request Scheduler

• Mayco S. Berghetti UFMS
• Fabrício B. Carvalho UFMS / UFMT
• Ronaldo A. Ferreira UFMS

DOI: https://doi.org/10.5753/sbrc.2024.1554

Abstract

Applications that serve millions of users over the Internet, such as social networks and online games, are typically decomposed into microservices to scale and provide low response times. Many of these applications process requests with high service-time dispersion, which leads to the Head-Of-Line Blocking problem in which requests with high service times block other requests and increase the average and tail latencies of the applications’ requests. This work presents AFP (Application Feedback Policy), a scheduling policy that mitigates this problem. Experimental results show that AFP processes up to 4.5× more workload than the best-known approach for SLOs of a few microseconds.

References

Ardelean, D. et al. (2018). Performance Analysis of Cloud Applications. In USENIX NSDI’18.

Barbette, T. et al. (2019). RSS++ Load and State-aware Receive Side Scaling. In ACM CoNEXT’19.

Belay, A. et al. (2012). Dune: Safe User-level Access to Privileged CPU Features. In USENIX OSDI’12.

Benson, T. et al. (2010). Network Traffic Characteristics of Data Centers in the Wild. In ACM IMC’10.

Carvalho, F. B. et al. (2022). Dyssect: Dynamic Scaling of Stateful Network Functions. In IEEE INFOCOM 2022.

Dean, J. and Barroso, L. A. (2013). The Tail at Scale. Communications of the ACM.

Delgado, P. et al. (2016). Job-aware Scheduling in Eagle: Divide and Stick to Your Probes. In ACM SoCC’16.

Demoulin, H. M. et al. (2021). When Idling is Ideal: Optimizing Tail-latency for Heavy-tailed Datacenter Workloads with Perséphone. In ACM SOSP’21.

Didona, D. and Zwaenepoel, W. (2019). Size-Aware Sharding for Improving Tail Latencies in In-Memory Key-Value Stores. In USENIX NSDI’19.

Foundation, L. (2024). The Linux Kernel. Disponível em [link].

Fried, J. et al. (2020). Caladan: Mitigating Interference at Microsecond Timescales. In USENIX OSDI’20.

Intel (2023). Intel® 64 and IA-32 Architectures Software Developer’s Volumes 3C: System Programming Guide. Seção 26.5.5.

Kaffes, K. et al. (2019). Shinjuku: Preemptive Scheduling for µsecond-scale Tail Latency. In USENIX NSDI’19.

McClure, S. et al. (2022). Efficient Scheduling Policies for Microsecond-Scale Tasks. In USENIX NSDI’22.

Nishtala, R. et al. (2013). Scaling Memcache at Facebook. In USENIX NSDI’13.

Ousterhout, A. et al. (2019). Shenango: Achieving High CPU Efficiency for Latency-Sensitive Datacenter Workloads. In USENIX NSDI’19.

Ousterhout, J. et al. (2015). The RAMCloud Storage System. ACM Transactions on Computer Systems (TOCS).

Peter, S. et al. (2015). Arrakis: The Operating System is the Control Plane. ACM Transactions on Computer Systems (TOCS).

Prekas, G. et al. (2017). Zygos: Achieving Low Tail Latency for Microsecond-Scale Networked Tasks. In ACM SOSP’17.

Qin, H. et al. (2018). Arachne: Core-Aware Thread Management. In USENIX OSDI’18.

Redis Ltd. (2023). Redis. Disponível em [link].

Schöne, R. et al. (2015). Wake-up Latencies for Processor Idle States on Current x86 Processors. Computer Science-Research and Development.

Sohil Mehta (2023). x86 User Interrupts support. Disponível em [link].

Wierman, A. and Zwart, B. (2012). Is Tail-Optimal Scheduling Possible? Operations Research.

Zhang, Z. et al. (2022). CRISP: Critical Path Analysis of Large-Scale Microservice Architectures. In USENIX ATC’22.