TCUDA: A QoS-based GPU Sharing Framework for Autonomous Navigation Systems

• Pangbo Sun TuSimple
• Hao Wu TuSimple
• Jiangming Jin TuSimple
• Ziyue Jiang TuSimple
• Yifan Gong TuSimple

Resumo

Autonomous navigation systems (ANS) consist of several software modules, such as sensing, perception, and planning to achieve traffic perception and fast decision making. These modules are required to process large amounts of data, such as images, in real-time. GPUs are commonly exploited on ANS to speed up data processing. As GPUs provides tremendous computation resources, it is common that one software module uses only partial GPU resources, leading to low GPU utilization and energy inefficiency. Traditionally, GPU sharing is a method to address this problem. However, GPU sharing is not supported on current embedded GPU platforms, which are widely used by ANS. Furthermore, GPU sharing challenges task Quality of Service (QoS) that requires task execution in a fixed latency. To achieve QoS requirement, we propose a progress bar scheduling policy to provide GPU tasks with QoS guarantee in GPU sharing environments. Based on this policy, a GPU sharing framework named TCUDA, is proposed to endow existing GPU tasks with QoS guarantee and improve GPU utilization. Finally, results show TCUDA reduces GPU task latency by 16.8% with QoS guarantee on an embedded GPU platform Xavier.